stjohn_piano 2018-01-15 no GOAL: List the tools that are available on Centos 6.9 Minimal. CONTENTS: - Goal - Contents - Downloadable Assets - Project Log DOWNLOADABLE ASSETS: asset available_tools_on_centos_69_minimal.txt available_tools_on_centos_69_minimal.txt ec499e608487b49db1b84a6f153356798ffa7cd28b1f10871f65b080985b87d8 asset display_bytes.py display_bytes.py df7bea971a5fa9e9858ac9ec561bdba206ff143529441d6327bc8a3d4a3c64b7 available_tools_on_centos_69_minimal.txt is a list of the available tools (commands, aliases, builtins, keywords) in the bash shell on Centos 6.9 Minimal. It can be searched using e.g. grep. display_bytes.txt is a Python 2.7 program that reads a file (specified in the code in the variable file_path) and displays the contents as ASCII characters with some substitutions that facilitate reading whitespace and unprintable character bytes. PROJECT LOG: I recently succeeded in article Installing_Centos_6.9_Minimal_on_Kalkin edgecase 28 installing Centos 6.9 Minimal on Kalkin . I would now like to know what tools are built into the bash shell on the Centos 6.9 Minimal operating system. Press the power button to boot Kalkin. Screen: Booting from local disk... Intel(R) Boot Agent GE v1.3.63 Copyright (C) 1997-2010, Intel Corporation Intel(R) Boot Agent PXE Base Code (PXE-2.1 build 089) Copyright (C) 1997-2010, Intel Corporation Initializing and establishing link... Screen: Booting from local disk... Intel(R) Boot Agent GE v1.3.63 Copyright (C) 1997-2010, Intel Corporation Intel(R) Boot Agent PXE Base Code (PXE-2.1 build 089) Copyright (C) 1997-2010, Intel Corporation PXE-E61: Media test failure, check cable PXE-M0F: Exiting Intel Boot Agent. Screen: Non-System disk or disk error replace and strike any key when ready Hm. Press the power button to turn Kalkin off. Press the power button to boot Kalkin. Centos boots. Reboot five more times. Centos boots each time. Odd. Press and hold the power button to turn Kalkin off. Press the power button to boot Kalkin. Screen: Press any key to enter the menu Booting CentOS 6 (2.6.32-696.e16.x86_64) in 3 seconds... [A booting screen is displayed] Screen: CentOS release 6.9 (Final) Kernel 2.6.32-696.el6.x86_64 on an x86_64 kalkin login: root Password: Last login: Thu Jan 4 00:43:02 on tty1 [root@kalkin ~]# [root@kalkin ~]# pwd /root [root@kalkin ~]# ls -1 anaconda-ks.cfg install.log install.log.syslog I'd like to record bash commands and their output into a text file. I could then mount a USB memory stick and copy this data directly into this project log, instead of transcribing it. Google "record terminal output". Fourth result: hyperlink http://askubuntu.com/questions/161935/how-do-i-log-all-input-and-output-in-a-terminal-session askubuntu.com/questions/161935/how-do-i-log-all-input-and-output-in-a-terminal-session Excerpt: Q: How do I keep logs of all the work done via terminal? For example, I want a record of all the work done on database via terminal. edited Jul 10 '12 at 12:31 Eliah Kagan asked Jul 10 '12 at 10:58 Ankit A: When you are ready to start recording a log file, type: script screen.log Now, until you stop the script, all input and output in the Terminal will be stored in screen.log. When you are done, just type: exit Your screen.log file will stored in the local directory. If you want to redirect it, use an absolute pathname such as ~/screen.log. This will do exactly what you are looking for. Source: Ubuntu Guide - How To Log Terminal Commands hyperlink http://ubuntuguide.net/how-to-log-terminal-commands-and-outputs-in-ubuntu [link] edited Nov 1 '16 at 23:34 Alan Thompson answered Jul 10 '12 at 11:21 xlukasx There's no guarantee that something from Ubuntu will work on Centos, but in principle the idea of the script utility is simple and useful enough that it's a good bet that it will exist on many versions of Linux. It probably predates Ubuntu. [root@kalkin ~]# which script /usr/bin/script [root@kalkin ~]# script --version script (util-linux-ng 2.17.2) [root@kalkin ~]# script screen.log Script started, file is screen.log [root@kalkin ~]# cat screen.log [root@kalkin ~]# exit exit Script done, file is screen.log [root@kalkin ~]# cat screen.log Script started on Fri 12 Jan 2018 01:16:45 PM GMT [root@kalkin ~]# cat screen.log [root@kalkin ~]# exit exit Script done on Fri 12 Jan 2018 01:17:10 PM GMT Excellent. I'll mount a memory stick and transfer this data. Insert memory stick (Sandisk, 8 GB) into Kalkin. [root@kalkin ~]# ls /mnt [root@kalkin ~]# usb 2-1.6: device descriptor read/64, error -110 usb 2-1.6: device descriptor read/64, error -110 usb 2-1.6: device descriptor read/8, error -110 usb 2-1.6: device descriptor read/8, error -110 usb 2-1.6: device descriptor read/8, error -110 usb 2-1.6: device descriptor read/8, error -110 hub 2-1:1.0: unable to enumerate USB device on port 6 Output hangs. Press enter. New prompt appears. Hm. Remove memory stick. Insert memory stick into Aineko (2008 Macbook). Open Disk Utility. Disk Utility reports that the memory stick's partition map scheme is Master Boot Record and its format is MS-DOS (FAT32). Eject and remove memory stick from Aineko. Hm. Let's try with a different memory stick (Kingston, 32 GB). Insert memory stick into Aineko (2008 Macbook). Open Disk Utility. Disk Utility reports that the Kingston memory stick's partition map scheme is Master Boot Record and its format is MS-DOS (FAT32). Eject and remove memory stick from Aineko. Insert this memory stick into Kalkin. [root@kalkin ~]# sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. [root@kalkin ~]# fdisk -l Disk /dev/sda: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000605ad Device Boot Start End Blocks Id System /dev/sda1 * 1 64 512000 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 64 243202 1953001472 8e Linux LVM Disk /dev/mapper/vg_kalkin-lv_root: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6257 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/mapper/vg_kalkin-lv_swap: 8413 MB, 8413773824 bytes 255 heads, 63 sectors/track, 1022 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/mapper/vg_kalkin-lv_home: 1937.8 GB, 1937768448000 bytes 255 heads, 63 sectors/track, 235586 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdg: 31.1 GB, 31051513856 bytes 255 heads, 63 sectors/track, 3775 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xaac1950c Device Boot Start End Blocks Id System /dev/sdg1 * 1 3776 30323712 c W95 FAT32 (LBA) Disk /dev/sdg is the expected size (~32 GB) for the memory stick. Mount the memory stick. [root@kalkin ~]# mkdir /mnt/test1 [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls -1 /mnt test1 [root@kalkin ~]# ls -1 /mnt/test1 CentOS-6.9-x86_64-minimal.iso The file CentOS-6.9-x86_64-minimal.iso is left over from a previous project. [root@kalkin ~]# rm /mnt/test1/CentOS-6.9-x86_64-minimal.iso rm: remove regular file `/mnt/test1/CentOS-6.9-x86_64-minimal.iso'? y [root@kalkin ~]# cp screen.log /mnt/test1/ [root@kalkin ~]# ls -1 /mnt/test1 screen.log [root@kalkin ~]# umount /dev/sdg1 Remove the memory stick. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. Open screen.log. It's empty. Hm. Eject and remove the memory stick from Aineko. Insert the memory stick into Kalkin. [root@kalkin ~]# sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls -1 /mnt/test1 screen.log [root@kalkin ~]# cat /mnt/test1/screen.log Why has the contents of screen.log not been preserved? I'll try copying to a specific filename instead of targeting a directory. [root@kalkin ~]# cp screen.log /mnt/test1/foo.txt [root@kalkin ~]# cat /mnt/test1/foo.txt Script started on Fri 12 Jan 2018 01:16:45 PM GMT [root@kalkin ~]# cat screen.log [root@kalkin ~]# exit exit Script done on Fri 12 Jan 2018 01:17:10 PM GMT [root@kalkin ~]# umount /dev/sdg1 [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# cat /mnt/test1/foo.txt Script started on Fri 12 Jan 2018 01:16:45 PM GMT [root@kalkin ~]# cat screen.log [root@kalkin ~]# exit exit Script done on Fri 12 Jan 2018 01:17:10 PM GMT Hm. It looks as though the data has been successfully copied onto the disk. [root@kalkin ~]# umount /dev/sdg1 Remove the memory stick from Kalkin. Insert the memory stick into Aineko. It mounts automatically and an icon appears on the Desktop. Open foo.txt. Contents (as shown in TextWrangler): Script started on Fri 12 Jan 2018 01:16:45 PM GMT [root@kalkin ~]# cat screen.log [root@kalkin ~]# exit exit Script done on Fri 12 Jan 2018 01:17:10 PM GMT I don't why there are extra newlines. Let's look at the raw bytes in foo.txt. I'll write some Python code to do this. Display Bytes in Python python_2.7.13 asset display_bytes.py [download this code] df7bea971a5fa9e9858ac9ec561bdba206ff143529441d6327bc8a3d4a3c64b7 #!/opt/local/bin/python file_path = "foo.txt" f = open(file_path) data = f.read() f.close() display_string="" for byte in data: # get the decimal integer value of the byte n = ord(byte) # these are the byte values for the printable characters of ASCII plus the newline, the tab, and the space. allowed_values = [9,10,32] + range(33,127) if n not in allowed_values: # non-printable ASCII bytes display_string += "[ord_%d]" % n elif n in [9,10,32]: # whitespace bytes if n == 9: # tab byte display_string += "[tab]\\t" elif n == 10: # new line byte display_string += "[newline]\\n" elif n == 32: display_string += " " else: # printable ASCII characters char = byte display_string += char print display_string Move foo.txt to Downloads directory. Save the Python code in the Downloads directory as display_bytes.py. aineko:~ stjohnpiano$ which python /opt/local/bin/python aineko:~ stjohnpiano$ python --version Python 2.7.13 aineko:~ stjohnpiano$ cd Downloads aineko:Downloads stjohnpiano$ chmod 700 display_bytes.py aineko:Downloads stjohnpiano$ ./display_bytes.py Script started on Fri 12 Jan 2018 01:16:45 PM GMT[newline] [root@kalkin ~]# cat screen.log[ord_13][newline] [root@kalkin ~]# exit[ord_13][newline] exit[ord_13][newline] [newline] Script done on Fri 12 Jan 2018 01:17:10 PM GMT[newline] There are [ord_13] bytes in all the locations where I saw an extra newline in TextWrangler's rendering of foo.txt. Looking at my ASCII reference table, I find that byte 13 is CR (carriage return). Note: The carriage return is often typed in e.g. Python strings as "\\r". Evidently TextWrangler displays carriage returns while cat ignores them. I'm slightly surprised that script outputs carriage return bytes. Delete foo.txt and screen.log from the memory stick. Delete foo.txt from the Downloads directory on Aineko. Eject and remove the memory stick from Aineko. I'll stop here for today. Shut down Kalkin. [root@kalkin ~]# halt Another day, another crack at the problem. Press the power button to boot Kalkin. Screen: Booting from local disk... Intel(R) Boot Agent GE v1.3.63 Copyright (C) 1997-2010, Intel Corporation Intel(R) Boot Agent PXE Base Code (PXE-2.1 build 089) Copyright (C) 1997-2010, Intel Corporation Initializing and establishing link... Screen: Booting from local disk... Intel(R) Boot Agent GE v1.3.63 Copyright (C) 1997-2010, Intel Corporation Intel(R) Boot Agent PXE Base Code (PXE-2.1 build 089) Copyright (C) 1997-2010, Intel Corporation PXE-E61: Media test failure, check cable PXE-M0F: Exiting Intel Boot Agent. Screen: Non-System disk or disk error replace and strike any key when ready Hm. Press Enter. Screen: Non-System disk or disk error replace and strike any key when ready Press the power button to turn Kalkin off. Press the power button to boot Kalkin. Centos boots. Press and hold the power button to turn Kalkin off. Reboot four more times. Centos boots each time. It's as if the first boot changes a setting somewhere, which is stored in a capacitor, the charge in which then fades when the computer is off for a reasonably long period of time. Press and hold the power button to turn Kalkin off. Press the power button to boot Kalkin. Screen: Press any key to enter the menu Booting CentOS 6 (2.6.32-696.e16.x86_64) in 3 seconds... [A booting screen is displayed] I want to look at this problem again: - Copying a file to a target directory (instead of a target filename) on a mounted memory stick results in an empty 0-byte file. Screen: CentOS release 6.9 (Final) Kernel 2.6.32-696.el6.x86_64 on an x86_64 kalkin login: root Password: Last login: Thu Jan 4 00:43:02 on tty1 [root@kalkin ~]# [root@kalkin ~]# ls -1 anaconda-ks.cfg install.log install.log.syslog screen.log [root@kalkin ~]# rm screen.log rm: remove regular file `screen.log'? y [root@kalkin ~]# echo "hello world" \> bar.txt [root@kalkin ~]# cat bar.txt hello world Insert memory stick (Kingston, 32 GB) into Kalkin. [root@kalkin ~]# sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. I'd like to find the device identifier of the inserted memory stick via reading something more concise than the output of fdisk -l. While article Installing_Centos_6.9_Minimal_on_Kalkin edgecase 28 installing Centos 6.9 Minimal on Kalkin , I discovered the blkid command. [root@kalkin ~]# blkid /dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4" /dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member" /dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4" /dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap" /dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4" /dev/sdg1: LABEL="KINGSTON" UUID="7195-020C" TYPE="vfat" The Kingston stick has the identifier /dev/sdg1. [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls -1 /mnt/test1 [root@kalkin ~]# cp bar.txt /mnt/test1 [root@kalkin ~]# cat /mnt/test1/bar.txt hello world Hm. Ah, maybe the trailing slash on the target directory makes a difference. I used one before when the cp command produced a 0-byte file. [root@kalkin ~]# cp bar.txt bar2.txt [root@kalkin ~]# cp bar2.txt /mnt/test1/ [root@kalkin ~]# cat /mnt/test1/bar2.txt hello world Hm. [root@kalkin ~]# umount /dev/sdg1 Remove the memory stick. Insert it into Aineko. It mounts automatically and an icon appears on the Desktop. aineko:~ stjohnpiano$ ls -1 /Volumes KINGSTON Macintosh HD Untitled aineko:~ stjohnpiano$ cd /Volumes/KINGSTON aineko:KINGSTON stjohnpiano$ ls aineko:KINGSTON stjohnpiano$ ls -1a . .. .Spotlight-V100 .Trashes ._.Trashes .fseventsd Hm. Neither file is present. Eject and remove the memory stick from Aineko. Insert the memory stick into Kalkin. [root@kalkin ~]# sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. Running blkid shows me that the identifier is again /dev/sdg1. [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls /mnt/test1 Neither file is present. Hm. Let's try again, but this time I'll dismount it and remount it without removing it from Kalkin. Hm. [some reading occurs here]. Wait a second, I've been using umount incorrectly. One should target the mountpoint directory, not the device identifier. [root@kalkin ~]# cat bar.txt hello world [root@kalkin ~]# cp bar.txt /mnt/test1 [root@kalkin ~]# cat /mnt/test1/bar.txt hello world [root@kalkin ~]# umount /mnt/test1 [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls /mnt/test1 bar.txt [root@kalkin ~]# cat /mnt/test1/bar.txt hello world It looks like the problem was caused by incorrect use of umount. [root@kalkin ~]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert the memory stick into Aineko. It mounts automatically and an icon appears on the Desktop. aineko:KINGSTON stjohnpiano$ ls aineko:KINGSTON stjohnpiano$ ls bar.txt aineko:KINGSTON stjohnpiano$ cat bar.txt hello world Excellent. Eject and remove memory stick from Aineko. Insert memory stick into Kalkin. [root@kalkin ~]# sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. [root@kalkin ~]# mount /dev/sdg1 /mnt/test1 [root@kalkin ~]# ls /mnt/test1 bar.txt [root@kalkin ~]# cat /mnt/test1/bar.txt hello world When I was article Checking_Kalkin's_hardware_details_using_Centos_6.9 edgecase 24 checking Kalkin's hardware details using a Centos 6.9 bootable USB stick , I discovered the compgen command, a bash builtin, which generates possible completion matches for words in the bash shell. If no words are provided, and it is given at least one option, it will list all possible words (given that option). I'm going to run compgen with various options in order to gather lists of the available commands on Centos 6.9 Minimal. I will use the memory stick to transfer the compgen results, recorded via script, to Aineko for inclusion in this log. [root@kalkin ~]# rm bar.txt bar2.txt rm: remove regular file `bar.txt'? y rm: remove regular file `bar2.txt'? y [root@kalkin ~]# cd /home [root@kalkin home]# mkdir work [root@kalkin home]# cd work [root@kalkin work]# compgen [a shutdown occurs here. I've rerun the appropriate commands and arrived back at this point.] [root@kalkin work]# script work.log [various commands are run here. compgen output will be pasted into this log later.] [root@kalkin work]# rm /mnt/test1/bar.txt rm: remove regular file `bar.txt'? y [root@kalkin work]# cp work.log /mnt/test1 [root@kalkin work]# ls /mnt/test1 work.log [root@kalkin work]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. The file work.log is present but it's 0 KB i.e. empty. Hm. Eject and remove memory stick from Aineko. Insert the stick into Kalkin. [root@kalkin work]# sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. I'll copy to a filename rather than to a directory. [root@kalkin work]# cp work.log /mnt/test1/work.log cp: overwrite `/mnt/test1/work.log'? y [root@kalkin work]# head -1 /mnt/test1/work.log Script started on Mon 15 2018 02:10:00 AM GMT [root@kalkin work]# cat /mnt/test1/work.log | wc -l 9072 [root@kalkin work]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. work.log is again 0 KB. Looking back at this log, it appears that when I followed this sequence: - unmount memory stick on Kalkin - mount memory stick on Kalkin - unmount memory stick on Kalkin and only then removed the memory stick and inserted it into Aineko, the data was preserved. Odd. Let's try that again. Eject and remove memory stick from Aineko. Insert the stick into Kalkin. [root@kalkin work]# sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 work.log [root@kalkin work]# cat /mnt/test1/work.log [root@kalkin work]# cp work.log /mnt/test1/work.log cp: overwrite `/mnt/test1/work.log'? y [root@kalkin work]# cat /mnt/test1/work.log | wc -l 9072 [root@kalkin work]# umount /mnt/test1 [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 work.log [root@kalkin work]# cat /mnt/test1/work.log | wc -l 9072 [root@kalkin work]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. work.log is present and its size is 131 KB. The unmount and remount step seems to do something that causes the data to be preserved on the memory stick. aineko:KINGSTON stjohnpiano$ ls work.log On opening the file in TextWrangler, I see the extra newlines caused by the inclusion of carriage return bytes. I'd like to remove the carriage return bytes from the output. Google "count occurrences of particular byte". Second result: hyperlink http://stackoverflow.com/questions/1603566/count-occurrences-of-a-char-in-plain-text-file stackoverflow.com/questions/1603566/count-occurrences-of-a-char-in-plain-text-file Excerpt: Q: Is there any way under linux/terminal to count, how many times the char f occurs in a plain text file? edited Oct 21 '09 at 21:24 Rob Hruska asked Oct 21 '09 at 20:56 cupakob A: [...] tr -cd f \< file | wc -c [...] edited Jan 29 '13 at 10:05 erik answered Jan 17 '13 at 0:33 user1985553 Comment: To count several characters, e.g. a, b and c : tr -cd abc \< file | wc -l. Skippy le Grand Gourou Apr 3 '17 at 13:26 Well, I know from running cat work.log | wc -l that work.log is 9072 lines long and therefore contains 9072 newline bytes. This is a data point. aineko:KINGSTON stjohnpiano$ tr -cd \\n \< work.log | wc -c 3116 aineko:KINGSTON stjohnpiano$ tr -cd \\ \< work.log | wc -c 482 aineko:KINGSTON stjohnpiano$ tr -cd n \< work.log | wc -c 3116 aineko:KINGSTON stjohnpiano$ tr -cd "\\n" \< work.log | wc -c 9072 Cool. Evidently there's some rules about quoting and backslash characters. aineko:KINGSTON stjohnpiano$ tr -cd a \< work.log | wc -c 4032 aineko:KINGSTON stjohnpiano$ tr -cd b \< work.log | wc -c 1431 aineko:KINGSTON stjohnpiano$ tr -cd ab \< work.log | wc -c 5463 4032 + 1431 = 5463 I'm not sure why this addition didn't hold for \\, n, and \\n, (not quoted, so treated as separate bytes?) in the earlier commands. Probably backslash is special. aineko:KINGSTON stjohnpiano$ tr -cd "\\r" \< work.log | wc -c 9069 9069 = 9072 - 3 This carriage return byte count matches the first script recording, in which the first line and the last two lines each had only a newline, instead of carriage return + newline. aineko:KINGSTON stjohnpiano$ man tr Excerpt from man page: NAME tr -- translate characters SYNOPSIS tr [-Ccsu] string1 string2 tr [-Ccu] -d string1 tr [-Ccu] -s string1 tr [-Ccu] -ds string1 string2 DESCRIPTION The tr utility copies the standard input to the standard output with substitution or deletion of selected characters. The following options are available: -C Complement the set of characters in string1, that is ``-C ab'' includes every character except for `a' and `b'. -c Same as -C but complement the set of values in string1. -d Delete characters in string1 from the input. [...] In the second synopsis form, the characters in string1 are deleted from the input. [...] The following conventions can be used in string1 and string2 to specify sets of characters: character Any character not described by one of the following conventions represents itself. \\character A backslash followed by certain special characters maps to special values. \\a \ \\b \ \\f \ \\n \ \\r \ \\t \ \\v \ A backslash followed by any other character maps to that character. [...] EXAMPLES The following examples are shown as given to the shell: Create a list of the words in file1, one per line, where a word is taken to be a maximal string of letters. tr -cs "[:alpha:]" "\\n" \< file1 [...] Strip out non-printable characters from file1. tr -cd "[:print:]" \< file1 [...] BSD July 23, 2004 Hm. From the man page descriptions, -C and -c appear to be the same option. I suspect that the description of -C should actually be "include only characters in the set of values in string1". In the examples, I note the use of double quotation marks around string1 and string2. Evidently they can be omitted if the input string consists of non-backspace-escaped characters. No mention of how to target the backslash character when using tr. Hm. "A backslash followed by any other character maps to that character." Perhaps in the command tr -cd \\ \< work.log | wc -c, the target string was actually "\\ ", i.e. the backslash and also the space following it, which would then be treated as a space. aineko:KINGSTON stjohnpiano$ tr -cd \\ \< work.log | wc -c 482 aineko:KINGSTON stjohnpiano$ tr -cd " " \< work.log | wc -c 482 Yup. It counted spaces. Further thoughts on "A backslash followed by any other character maps to that character": - targeting \\n without quotation marks actually counted the occurrences of n by itself, which is why the results for \\n and n were the same. - to target a backslash, I should probably use "\\\\". In the command: tr -cd "\\r" \< work.log tr proceeds through each line in the input (which in this case is the file work.log), copying the result of evaluating that line to the standard output. -c selects every byte that is not "\\r". -d deletes every selected byte. Adding | wc -c to this command uses the wc (which stands for "word count") utility to count each character that tr sent to the standard output. So, in this result: aineko:KINGSTON stjohnpiano$ tr -cd "\\r" \< work.log | wc -c 9069 I can see that the carriage return byte "\\r" occurs 9069 times in work.log. If I can correctly count a byte type, this means I can select it, which means I can delete it. aineko:KINGSTON stjohnpiano$ wc -c work.log 124870 work.log There are 124870 total in work.log. I'll leave out the -c option, so instead tr should delete every instance of "\\r" and copy all other bytes to the standard output. aineko:KINGSTON stjohnpiano$ tr -d "\\r" \< work.log | wc -c 115801 124870 - 9069 = 115801 115801 is the number of bytes expected after removing all instances of "\\r". Finally, I'll redirect the output into a file. This file will be the same as work.log, except that it will not contain any carriage return bytes. aineko:KINGSTON stjohnpiano$ tr -d "\\r" \< work.log \> work2.log aineko:KINGSTON stjohnpiano$ wc -c work2.log 115801 work2.log Excellent. Here are the relevant excerpts from work2.log, reordered and edited a bit. There are a few strange renderings of the command inputs e.g. compgen -a [Kc and \\compgen -c | sort. I'm not sure why these have occurred. They are in work.log as well as work2.log. I'll remove them manually. I've rerun the compgen commands on Kalkin, piping the output into less, and checked that the output matches that shown here. I didn't find that any of the output was rendered differently, but it's good to be sure. Descriptions of the compgen commands: compgen -c will list all the commands you could run. compgen -a will list all the aliases you could run. compgen -b will list all the builtins you could run. compgen -k will list all the keywords you could run. compgen -A function will list all the functions you could run. compgen -A function -abck will list all the above in one go. Script started on Mon 15 Jan 2018 02:10:00 AM GMT [root@kalkin work]# compgen --version bash: compgen: --: invalid option compgen: usage: compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [word] [root@kalkin work]# compgen -a cp l. ll ls mv rm which [root@kalkin work]# compgen -b . : [ alias bg bind break builtin caller cd command compgen complete compopt continue declare dirs disown echo enable eval exec exit export false fc fg getopts hash help history jobs kill let local logout mapfile popd printf pushd pwd read readarray readonly return set shift shopt source suspend test times trap true type typeset ulimit umask unalias unset wait [root@kalkin work]# compgen -k if then else elif fi case esac for select while until do done in function time { } ! [[ ]] coproc [root@kalkin work]# compgen -A function [root@kalkin work]# compgen -c | sort : ! . [ [ [[ ]] { } addftinfo addgnupghome addpart addr2line adduser agetty alias alternatives anacron applygnupgdefaults ar arch arp arpd arping arping as attr audispd auditctl auditd augenrules aulast aulastlog aureport ausearch ausyscall authconfig authconfig-tui autrace auvirt avcstat awk awk badblocks base64 basename bash bashbug-64 berkeley_db_svc bg bind blkdeactivate blkdiscard blkid blockdev brcm_iscsiuio brctl break bridge build-locale-archive builtin bunzip2 bzcat bzcmp bzdiff bzgrep bzip2 bzip2recover bzless bzmore cacertdir_rehash cache_check cache_check cache_dump cache_dump cache_metadata_size cache_metadata_size cache_repair cache_repair cache_restore cache_restore cal ca-legacy caller capsh captoinfo case cat catchsegv cbq cciss_id cd certutil cfdisk c++filt chacl chage chattr chcon chcpu checkmodule checkpolicy chfn chgrp chkconfig chmod chown chpasswd chroot chrt chsh chvt cksum clear clock clockdiff cmp cmsutil col colcrt colrm column comm command compgen complete compopt consoletype continue coproc cp cp cpio cracklib-check cracklib-format cracklib-packer cracklib-unpacker create-cracklib-dict crlutil crond crontab cryptsetup csplit ctrlaltdel curl cut cut dash date db_archive db_checkpoint db_codegen db_deadlock db_dump db_dump185 db_hotbackup db_load db_printlog db_recover db_stat db_upgrade dbus-binding-tool db_verify dd ddate deallocvt debugfs declare delpart depmod df dhclient dhclient-script diff diff3 dir dircolors dirname dirs disown dmesg dmeventd dmsetup dmstats dnsdomainname do domainname done dracut du dumpe2fs dumpkeys e2freefrag e2fsck e2image e2label e2undo echo echo efibootmgr egrep elif else enable env env eqn eqn2graph era_check era_check era_dump era_dump era_invalidate era_invalidate era_restore era_restore esac ether-wake ethtool ethtool eval ex exec exit expand export expr factor faillock fallocate false false fc fdformat fdisk fg fgconsole fgrep fi file filefrag find find findfs findmnt fipscheck fipshmac fixfiles flock floppy fmt fold for free fsadm fsck fsck.cramfs fsck.ext2 fsck.ext3 fsck.ext4 fsck.ext4dev fsck.xfs fsfreeze fstab-decode fstrim function fuser fusermount fusermount gawk gawk gdbus gencat genhomedircon genhostid geqn getcap getconf getenforce getent getfacl getfattr getkey getkeycodes getopt getopts getpcaps getsebool gindxbib gio-querymodules-64 glib-compile-schemas glibc_post_upgrade.x86_64 glookbib gmake gneqn gnroff gpasswd gpg gpg2 gpg-agent gpgconf gpg-connect-agent gpg-error gpgkey2ssh gpgparsemail gpgsplit gpgv gpgv2 gpg-zip gpic gprof grefer grep grn grodvi groff groffer grog grolbp grolj4 grops grotty groupadd groupdel groupmems groupmod groups grpck grpconv grpunconv grub grubby grub-crypt grub-install grub-md5-crypt grub-terminfo gsoelim gtar gtbl gtroff gunzip gunzip gzexe gzip gzip halt hash head help hexdump history hostid hostname hpftodit hwclock hwclock i386 iconv iconvconfig iconvconfig.x86_64 id idn if ifcfg ifconfig ifdown ifenslave ifup igawk in indxbib info infocmp infokey infotocap init initctl insmod insmod.static install install-info installkernel ionice ip ip6tables ip6tables-1.4.7 ip6tables-multi ip6tables-multi-1.4.7 ip6tables-restore ip6tables-restore-1.4.7 ip6tables-save ip6tables-save-1.4.7 ipcalc ipcmk ipcrm ipcs ipmaddr iptables iptables-1.4.7 iptables-multi iptables-multi-1.4.7 iptables-restore iptables-restore-1.4.7 iptables-save iptables-save-1.4.7 iptables-xml iptables-xml-1.4.7 iptunnel iscsiadm iscsid iscsi-iname iscsistart iscsiuio isosize jobs join kbd_mode kbdrate kill kill kill killall killall5 kpartx l. last lastb lastlog lchage lchfn lchsh ld ldattach ldconfig ldd less lessecho lesskey lesspipe.sh let lgroupadd lgroupdel lgroupmod lid link linux32 linux64 lkbib ll ln lnewusers lnstat loadkeys load_policy load_policy loadunimap local locale localedef logger logger login logname logout logrotate logsave lokkit look lookbib losetup lpasswd ls ls lsattr lsblk lscpu lsinitrd lslogins lsmod lua luac luseradd luserdel lusermod lvchange lvconvert lvcreate lvdisplay lvextend lvm lvmchange lvmconf lvmconfig lvmdiskscan lvmdump lvmetad lvmsadc lvmsar lvreduce lvremove lvrename lvresize lvs lvscan m4 mailq mailq.postfix make MAKEDEV mapfile mapscrn matchpathcon matchpathcon mbchk mcookie md5sum mdadm mdmon mesg mii-diag mii-tool mingetty mkdict mkdir mke2fs mkfifo mkfs mkfs.cramfs mkfs.ext2 mkfs.ext3 mkfs.ext4 mkfs.ext4dev mkfs.xfs mkhomedir_helper mkinitrd mklost+found mknod mksock mkswap mktemp modinfo modprobe modutil more mount mount.fuse mountpoint mount.tmpfs mpathconf mpathpersist multipath multipathd mv mv namei nameif neqn netreport netstat newaliases newaliases.postfix newgrp new-kernel-pkg newusers nice nisdomainname nl nm nohup nologin nproc nroff nsenter nstat objcopy objdump od oldfind open open_init_pty openssl openvt p11-kit packer pam_console_apply pam_tally2 pam_timestamp_check partx passwd paste pathchk pcregrep pcretest pdata_tools pdata_tools peekfd pfbtops pgawk pgrep pic pic2graph pidof pinentry pinentry-curses ping ping6 ping6 pinky pivot_root pk12util pkg-config pkill plipconfig pluginviewer plymouth plymouth plymouthd plymouth-set-default-theme pmap popd postalias postcat postconf postdrop postfix post-grohtml postkick postlock postlog postmap postmulti postqueue postsuper poweroff ppp-watch pr pre-grohtml printenv printf printf ps psfaddtable psfgettable psfstriptable psfxtable pstree pstree.x11 ptx pushd pvchange pvck pvcreate pvdisplay pvmove pvremove pvresize pvs pvscan pwck pwconv pwd pwd pwdx pwunconv pydoc python python2 python2.6 raid-check ranlib raw rdisc read readarray readelf readlink readlink readonly readprofile reboot refer reload rename renice reset resize2fs resizecons restart restorecon restorecond return rev rhgb-client rm rm rmail rmail.postfix rmdir rmmod route rpcgen rpm rpm2cpio rpmdb rpmquery rpmsign rpmverify rsyslogd rtacct rtcwake rtmon runcon run_init runlevel run-parts runuser rvi rview saslauthd sasldblistusers2 saslpasswd2 scp script scriptreplay scsi_id sdiff secon securetty sed sedismod sedispol select selinuxconlist selinuxdefcon selinuxenabled semodule semodule_deps semodule_expand semodule_link semodule_package sendmail sendmail.postfix seq service sestatus set setarch setcap setenforce setfacl setfattr setfiles setfont setkeycodes setleds setmetamode setsebool setsid setsysfont setterm setup-nsssysinit.sh sfdisk sftp sg sh sha1sum sha224sum sha256sum sha384sum sha512sum shift shopt showconsolefont showkey shred shuf shutdown signtool signver size skill slabtop slattach sleep sln slogin smtp-sink smtp-source snice soelim sort source split sprof sqlite3 ss ssh ssh-add ssh-agent ssh-copy-id sshd ssh-keygen ssh-keyscan ssltap start start_udev stat status stdbuf stop strings strip stty su sudo sudoedit sudoreplay sulogin sum sushell suspend swapoff swapon switch_root sync sysctl sys-unconfig tabs tac tail tailf tar taskset tbl tc tee telinit test test testsaslauthd tfmtodit then thin_check thin_check thin_delta thin_delta thin_dump thin_dump thin_ls thin_ls thin_metadata_size thin_metadata_size thin_repair thin_repair thin_restore thin_restore thin_rmap thin_rmap thin_trim thin_trim tic time timeout times tload toe togglesebool top touch tput tr tracepath tracepath tracepath6 tracepath6 trap troff true true truncate tset tsort tty tune2fs tunelp type typeset tzdata-update tzselect udevadm udevd ul ulimit ulockmgr_server ulockmgr_server umask umount unalias uname unexpand unicode_start unicode_stop uniq unix_chkpwd unix_update unlink unset unshare until update-alternatives update-ca-trust update-mime-database uptime urlgrabber useradd userdel usermod usernetctl users usleep utmpdump uuidgen vdir vgcfgbackup vgcfgrestore vgchange vgck vgconvert vgcreate vgdisplay vgexport vgextend vgimport vgimportclone vgmerge vgmknodes vgreduce vgremove vgrename vgs vgscan vgsplit vi view vigr vipw visudo vmstat w wait wall watch watchgnupg wc weak-modules whereis which which while whiptail who whoami wipefs write x86_64 xargs xfs_admin xfs_bmap xfs_check xfs_copy xfs_db xfs_estimate xfs_freeze xfs_fsr xfs_growfs xfs_info xfs_io xfs_logprint xfs_mdrestore xfs_metadump xfs_mkfile xfs_ncheck xfs_quota xfs_repair xfs_rtcp xmlcatalog xmllint xmlwf yes ypdomainname yum zcat zcmp zdiff zdump zegrep zfgrep zforce zgrep zic zless zmore znew zsoelim Script done on Mon 15 Jan 2018 02:19:42 AM GMT After listing the available commands with compgen, I then did some exploration of the filesystem, using e.g. ls /usr/bin, etc. Unfortunately the output produced by script was rather strange, so I haven't included any here, except for the excerpt below that demonstrates the issue: Excerpt: [m[root@kalkin work]# ls -1 /bin [K [K [K [0m [01;34mbin [0m [01;34mboot [0m [01;34mdev [0m [01;34metc [0m [01;34mhome [0m [01;34mlib [0m [01;34mlib64 [0m [01;34mlost+found [0m [01;34mmedia [0m [01;34mmnt [0m [01;34mopt [0m [01;34mproc [0m [01;34mroot [0m [01;34msbin [0m [01;34mselinux [0m [01;34msrv [0m [01;34msys [0m [30;42mtmp [0m [01;34musr [0m [01;34mvar [0m [m[root@kalkin work]# ls -1 /boot config-2.6.32-696.el6.x86_64 [0m [01;34mefi [0m [01;34mgrub [0m initramfs-2.6.32-696.el6.x86_64.img [01;34mlost+found [0m [01;31msymvers-2.6.32-696.el6.x86_64.gz [0m System.map-2.6.32-696.el6.x86_64 [01;32mvmlinuz-2.6.32-696.el6.x86_64 [0m Last thing: Create a searchable list of the available tools (commands, aliases, builtins, keywords) in the bash shell on Centos 6.9 Minimal. Manually copy the outputs of: - compgen -a - compgen -b - compgen -k - compgen -c into a new file. Remove input lines and empty lines. Save it as tmp.txt in Downloads directory on Aineko. I'll sort tmp.txt and remove duplicate entries. aineko:KINGSTON stjohnpiano$ cd ~/Downloads aineko:Downloads stjohnpiano$ ls -1 display_bytes.py tmp.txt aineko:Downloads stjohnpiano$ sort -u \< tmp.txt \> tmp2.txt aineko:Downloads stjohnpiano$ mv tmp2.txt available_tools_on_centos_69_minimal.txt aineko:Downloads stjohnpiano$ rm -f tmp.txt I'll add available_tools_on_centos_69_minimal.txt and display_bytes.py as downloadable assets at the beginning of this article. Tidy up: - On the Kingston memory stick, delete work.log and work2.log. Changes from the original text: - I have not always preserved the format of any excerpts from webpages on other sites (e.g. not preserving the original bold/italic styles, changing the list structures, not preserving hyperlinks). - I have not always preserved the format of any computer output (e.g. from running bash commands). Examples: Setting input lines in bold text, adding/removing newlines in order to make a sequence of commands easier to read, using hyphens for lists and sublists instead of indentation, breaking wide tables into consecutive sections.