stjohn_piano 2018-01-25 no GOAL: Find a way to unmount a USB memory stick from Kalkin that preserves the data that has been written to it but does not require extra mount-unmount steps. CONTENTS: - Goal - Contents - Brief Summary - Representative Sample of Commands Used - Project Log BRIEF SUMMARY: I discovered that the problem was due to a faulty memory stick. Using another memory stick, this command sequence worked without any trouble: [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# cp hello.txt /mnt/test1/hello.txt [root@kalkin work]# umount /mnt/test1 Before running the command sequence above, insert the memory stick into the computer and run the command blkid to identify its device name. Use this device name in the mount command. Note: It is good practice to use the mountpoint directory as the argument for the umount command (instead of the device name), as the device may be mounted on multiple mountpoint directories. REPRESENTATIVE SAMPLE OF COMMANDS USED: [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 [root@kalkin work]# cp work.log /mnt/test1/work.log [root@kalkin work]# umount /mnt/test1 [root@kalkin ~]# mount [root@kalkin ~]# blkid [root@kalkin ~]# cd /home/work [root@kalkin work]# rm work.log [root@kalkin work]# echo "hello world" \> hello.txt [root@kalkin work]# cat hello.txt [root@kalkin work]# ls -a /mnt/test1 aineko:~ stjohnpiano$ ls -1 /Volumes/ aineko:HP v150w stjohnpiano$ ls -l aineko:HP v150w stjohnpiano$ ls -lh aineko:HP v150w stjohnpiano$ wc -l * aineko:HP v150w stjohnpiano$ head -1 kjv_bible.txt [root@kalkin work]# rm -f /mnt/test1/* PROJECT LOG: I encountered a problem with saving data to a USB memory stick while article Listing_available_tools_on_Centos_6.9_Minimal edgecase 30 listing available tools on Centos 6.9 Minimal . I had inserted a USB memory stick into Kalkin and mounted it to a mountpoint directory ( /mnt/test1 ). I then attempted to write a file ( work.log ) to this directory, which contained data that I wanted to transfer to another computer (Aineko). When the memory stick was unmounted and removed from Kalkin, and then inserted into Aineko and mounted, the file was 0 KB. No data had been preserved. Here is a relevant excerpt from the earlier project: Remove the memory stick from Kalkin. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. work.log is again 0 KB. Looking back at this log, it appears that when I followed this sequence: - unmount memory stick on Kalkin - mount memory stick on Kalkin - unmount memory stick on Kalkin and only then removed the memory stick and inserted it into Aineko, the data was preserved. Odd. Let's try that again. Eject and remove memory stick from Aineko. Insert the stick into Kalkin. [root@kalkin work]# sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press enter. New prompt appears. [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 work.log [root@kalkin work]# cat /mnt/test1/work.log [root@kalkin work]# cp work.log /mnt/test1/work.log cp: overwrite `/mnt/test1/work.log'? y [root@kalkin work]# cat /mnt/test1/work.log | wc -l 9072 [root@kalkin work]# umount /mnt/test1 [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 work.log [root@kalkin work]# cat /mnt/test1/work.log | wc -l 9072 [root@kalkin work]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert the stick into Aineko. It mounts automatically and an icon appears on the Desktop. work.log is present and its size is 131 KB. The unmount and remount step seems to do something that causes the data to be preserved on the memory stick. Key sentence: "The unmount and remount step seems to do something that causes the data to be preserved on the memory stick." I would like to remove these extra unmount and mount steps from my save-data-to-USB-stick sequence. If possible, I would also like to discover a good explanation for why the single unmount step did not work by itself. Google "unmount file not saved centos". Excerpt from: hyperlink http://www.ibm.com/developerworks/library/l-lpic1-104-3/index.html www.ibm.com/developerworks/library/l-lpic1-104-3/index.html Author: Ian Shields Published on October 20, 2010 Updated: January 27, 2016 Control mounting and unmounting of filesystems [...] The Linux filesystem is one big tree rooted at /, and yet we have filesystems on different devices and partitions. How do we resolve this apparent incongruity? The root (/) filesystem is mounted as part of the initialization process. Each of the other filesystems that you create is not usable by your Linux system until it is mounted at a mount point. [...] Unless otherwise noted, the examples in this tutorial use CentOS 6, with a 2.6.32-504 kernel. [...] A mount point is simply a directory where the filesystem on a device is grafted into the tree. Mounting is the process of making the filesystem on the device accessible. For example, you might mount filesystems on hard drive partitions as /boot, /tmp, or /home, and you might mount the filesystem on a floppy drive as /mnt/floppy and the filesystem on a CD-ROM as /media/cdrom1. As you can see, mount points may be in the root directory, or in a subdirectory farther down the tree. Besides filesystems on partitions, floppy disks, and CDs, there are other types of filesystems. The tmpfs filesystem is a virtual memory filesystem. You can also mount filesystems from one system on another system using a networked filesystem such as NFS or AFS. You can even create a file in an existing filesystem and format that as a, possibly different, kind of filesystem and mount that too. This is often done with images of optical media, where you perhaps download an ISO CD or DVD image, then mount that file rather than burning it to real media. Swap space in a file rather than a dedicated swap partition is another example. While the mount process actually mounts the filesystem on some device (or other resource), it is common to simply say that you "mount the device", which is understood to mean "mount the filesystem on the device". [...] When you mount a filesystem over an existing directory, the files on the filesystem you are mounting become the files and subdirectories of the mount point. If the mount point directory already contained files or subdirectories, they are not lost, but are no longer visible until the mounted filesystem is unmounted, at which point they become visible again. It is a good idea to avoid this problem by using only empty directories as mount points. [...] Usually, the mount command will automatically detect the type of filesystem being mounted. Occasionally you may need to specify the filesystem type explicitly using the -t option as shown in Listing 2. Listing 2. Mounting with explicit filesystem type [root@attic4-cent ~]# mount -t vfat /dev/sda3 /dos To see what filesystems are mounted, use the mount command with no parameters. [...] You can also view similar information by displaying /proc/mounts or /etc/mtab, both of which contain information about mounted filesystems. [...] All mounted filesystems are usually unmounted automatically by the system when it is rebooted or shut down. When a filesystem is unmounted, any cached filesystem data in memory is flushed to the device. You may also unmount filesystems manually. Indeed, you should do this when removing writable media such as diskettes or USB drives or memory keys. Use the unmount command to unmount the filesystem, specifying either the device name or mount point as an argument. This listing shows how to unmount /dos, then remount it and unmount again using the device name. Listing 8. Unmounting filesystems [root@attic4-cent ~]# umount /dos [root@attic4-cent ~]# mount /dev/sda3 /dos [root@attic4-cent ~]# umount /dev/sda3 After a filesystem is unmounted, any files in the directory used for the mount point are visible again. If you attempt to unmount a filesystem while a process has open files on that filesystem, you will see an error message. [...] Note: You should always unmount removable drives or media before disconnecting the drive or attempting to remove the media. Failure to do so may result in loss of data that has not yet been written to the device. Key sentence: "When a filesystem is unmounted, any cached filesystem data in memory is flushed to the device." I also note that either the device name or the mountpoint directory can be used as an argument for umount. Excerpt from: hyperlink http://linux.die.net/man/8/umount linux.die.net/man/8/umount The umount command detaches the file system(s) mentioned from the file hierarchy. A file system is specified by giving the directory where it has been mounted. Giving the special device on which the file system lives may also work, but is obsolete, mainly because it will fail in case this device was mounted on more than one directory. Ok. I'll make a habit of using the mountpoint directory as an argument for umount. I think that my problem might have been caused by a flush that didn't happen. When I wrote the file to the memory stick, this operation was completed in system memory (RAM). Checking the contents of the file would cause a query to the filesystem in RAM, which would contain the file at the right location and return its contents without any trouble. Then, when I ran the unmount command, the stick was unmounted, but the data had not been flushed to it from the RAM first. When I inserted the memory stick into Aineko, the file existed, but was empty. Running the mount-unmount steps after the initial unmount probably caused the flush to happen properly, for some reason that I don't understand. Let's try using a different memory stick. Press the power button to boot Kalkin. Centos boots. Log in. [root@kalkin ~]# mount /dev/mapper/vg_kalkin-lv_root on / type ext4 (rw) proc on /proc type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0") /dev/sda1 on /boot type ext4 (rw) /dev/mapper/vg_kalkin-lv_home on /home type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) New memory stick: HP, 32 GB. Insert memory stick into Aineko. It mounts automatically and an icon appears on the Desktop. Open Disk Utility. Disk Utility shows that the format of the disk is MS-DOS (FAT32) and its partition map scheme is MBR (Master Boot Record). The name of the single partition is "HP V150W". Eject and remove the disk from Aineko. Insert the disk into Kalkin. [root@kalkin ~]# sd 7:0:0:0: [sdg] Assuming drive cache: write through sd 7:0:0:0: [sdg] Assuming drive cache: write through sd 7:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press Enter. New prompt appears. [root@kalkin ~]# blkid /dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4" /dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member" /dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4" /dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap" /dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4" /dev/sdg1: LABEL="HP v150w" UUID="81A8-0559" TYPE="vfat" The device name of the second memory stick is /dev/sdg1. Running the mount command produces the same output as before. The device is visible to the operating system but is not mounted onto the filesystem tree. Let's move to my work directory, which I made earlier. [root@kalkin ~]# cd /home/work [root@kalkin work]# ls work.log [root@kalkin work]# rm work.log rm: remove regular file `work.log'? y [root@kalkin work]# echo "hello world" \> hello.txt [root@kalkin work]# ls hello.txt [root@kalkin work]# cat hello.txt hello world I removed work.log because it was used in a previous project and I no longer need it. I'm going to use hello.txt as my test file. Let's check the existing mountpoint directory and confirm that it's empty. [root@kalkin work]# ls /mnt test1 [root@kalkin work]# ls /mnt/test1 [root@kalkin work]# ls -a /mnt/test1 . .. I'll mount the second memory stick at the mountpoint directory, copy the test file to it, and unmount it. [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls /mnt/test1 [root@kalkin work]# cp hello.txt /mnt/test1/hello.txt [root@kalkin work]# umount /mnt/test1 Remove the memory stick from Kalkin. Insert it into Aineko. It mounts automatically and an icon appears on the Desktop. On Aineko: aineko:~ stjohnpiano$ ls /Volumes/ HP v150w Macintosh HD Untitled aineko:~ stjohnpiano$ ls -1 /Volumes/ HP v150w Macintosh HD Untitled aineko:~ stjohnpiano$ cd /Volumes/HP\\ v150w/ aineko:HP v150w stjohnpiano$ ls hello.txt aineko:HP v150w stjohnpiano$ cat hello.txt hello world Hm. It worked. Perhaps the original memory stick was faulty in some way. Perhaps the large size of work.log was a problem. It was 9072 lines and 131 KB. I should have kept work.log to use as a test file. I have a text file copy of the KJV Bible (downloaded from Project Gutenberg) on Aineko. Copy it to the memory stick. aineko:HP v150w stjohnpiano$ ls -l total 8544 -rwxrwxrwx 1 stjohnpiano staff 12 Jan 24 2018 hello.txt -rwxrwxrwx@ 1 stjohnpiano staff 4352552 Apr 27 2016 kjv_bible.txt aineko:HP v150w stjohnpiano$ ls -lh total 8544 -rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt -rwxrwxrwx@ 1 stjohnpiano staff 4.2M Apr 27 2016 kjv_bible.txt aineko:HP v150w stjohnpiano$ wc -l * 1 hello.txt 100304 kjv_bible.txt 100305 total aineko:HP v150w stjohnpiano$ head -1 kjv_bible.txt The Project Gutenberg EBook of The King James Bible It's 100304 lines and 4.2 MB. Eject and remove the memory stick from Aineko (close Terminal window first to allow ejection). I'm going to copy the large bible text file to Kalkin, delete it from the memory stick, and then copy it back to the memory stick in order to test the umount flush. Insert memory stick into Kalkin. [root@kalkin work]# sd 8:0:0:0: [sdg] Assuming drive cache: write through sd 8:0:0:0: [sdg] Assuming drive cache: write through sd 8:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press Enter. New prompt appears. [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls -1 /mnt/test1 hello.txt kjv_bible.txt [root@kalkin work]# cp /mnt/test1/kjv_bible.txt bible.txt [root@kalkin work]# ls -1 bible.txt hello.txt [root@kalkin work]# rm -f /mnt/test1/* [root@kalkin work]# ls -1 /mnt/test1 [root@kalkin work]# umount /mnt/test1 Unplug and reinsert memory stick into Kalkin. [root@kalkin work]# sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through sd 9:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press Enter. New prompt appears. [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls -1 /mnt/test1 [root@kalkin work]# cp bible.txt /mnt/test1/bible.txt [root@kalkin work]# ls -1 /mnt/test1 bible.txt [root@kalkin work]# head -1 /mnt/test1/bible.txt The Project Gutenberg EBook of The King James Bible [root@kalkin work]# wc -l /mnt/test1/bible.txt 100304 /mnt/test1/bible.txt [root@kalkin work]# ls -lh /mnt/test1/bible.txt -rwxr-xr-x. 1 root root 4.2M Jan 24 12:47 /mnt/test1/bible.txt [root@kalkin work]# umount /mnt/test1 Remove memory stick from Kalkin. Insert it into Aineko. It mounts automatically and an icon appears on the Desktop. On Aineko: aineko:~ stjohnpiano$ cd /Volumes/HP\\ v150w/ aineko:HP v150w stjohnpiano$ ls bible.txt aineko:HP v150w stjohnpiano$ ls -lh total 8512 -rwxrwxrwx 1 stjohnpiano staff 4.2M Jan 24 2018 bible.txt aineko:HP v150w stjohnpiano$ head -1 bible.txt The Project Gutenberg EBook of The King James Bible aineko:HP v150w stjohnpiano$ wc -l bible.txt 100304 bible.txt Hm. No problem copying a large file. Close the Terminal window. Eject and remove the memory stick from Aineko. Ok. Let's try again with the original memory stick (Kingston, 32 GB). I'll try to use it to transfer the hello.txt file. Insert the first memory stick into Aineko. It doesn't contain any files. Open Disk Utility. Disk Utility shows that the format of the disk is MS-DOS (FAT32) and its partition map scheme is MBR (Master Boot Record). The name of the single partition is "KINGSTON". Eject and remove the disk from Aineko. Insert the disk into Kalkin. [root@kalkin work]# sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through sd 10:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press Enter. New prompt appears. [root@kalkin work]# blkid /dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4" /dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member" /dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4" /dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap" /dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4" /dev/sdg1: LABEL="KINGSTON" UUID="7195-020C" TYPE="vfat" [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls -1 /mnt/test1 [root@kalkin work]# cp hello.txt /mnt/test1/hello.txt [root@kalkin work]# ls -1 /mnt/test1 hello.txt [root@kalkin work]# cat /mnt/test1/hello.txt hello world [root@kalkin work]# umount /mnt/test1 Remove memory stick from Kalkin. Insert it into Aineko. It mounts automatically and an icon appears on the Desktop. On Aineko: aineko:~ stjohnpiano$ ls -1 /Volumes/ KINGSTON Macintosh HD Untitled aineko:~ stjohnpiano$ cd /Volumes/KINGSTON/ aineko:KINGSTON stjohnpiano$ ls hello.txt aineko:KINGSTON stjohnpiano$ cat hello.txt aineko:KINGSTON stjohnpiano$ ls -lh total 32 -rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt Hm. 12 KB but it contains no text data. Let's try reformatting the memory stick. Close the Terminal window. Open Disk Utility. In Disk Utility, select the memory stick (it shows up as "Kingston DataTraveler 2.0 Media"). Click Erase tab. - Format = MS-DOS (FAT) - Name = KINGSTON Click Erase. Pop-up dialogue box opens with this message: Are you sure you want to erase the disk "Kingston DataTraveler 2.0 Media"? Erasing a disk deletes all data on all its partitions. This disk has 1 partition: - "Kingston" and two buttons: Cancel and Erase. Click Erase. Click Partition tab. Under Volume Scheme, choose 1 Partition. Click Options and choose Master Boot Record partition scheme. Under Volume Information, set these values: - Name = KINGSTON - Format = MS-DOS (FAT) - Size = 31.05 Click Apply. Pop-up dialogue box opens with this message: Are you sure you want to partition the disk "Kingston DataTraveler 2.0 Media"? Partitioning this disk will delete all data on this disk. This partition will be erased: "KINGSTON" This partition will be added: "KINGSTON" and two buttons: Cancel and Partition. Click Partition. Quit Disk Utility. Eject KINGSTON disk (right-click icon on Desktop and choose Eject). Remove the KINGSTON memory stick from Aineko. Insert the disk into Kalkin. [root@kalkin work]# sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through sd 11:0:0:0: [sdg] Assuming drive cache: write through Output hangs. Press Enter twice. New prompt appears. Note: I had to press Enter twice (the first press had no visible effect). This happened before in this project but I had thought I hadn't been pressing the key hard enough the first time. [root@kalkin work]# blkid /dev/sda1: UUID="e0b18dd5-f392-42ca-bbf1-89a1a0229e38" TYPE="ext4" /dev/sda2: UUID="rWbUY4-wwiQ-J2Vp-n0F5-bngF-G8yb-HWQyEv" TYPE="LVM2_member" /dev/mapper/vg_kalkin-lv_root: UUID="508692c4-54da-4a93-b2e2-6f1584e74afe" TYPE="ext4" /dev/mapper/vg_kalkin-lv_swap: UUID="707ee542-6bfe-453d-b43c-bcfc58c59b6f" TYPE="swap" /dev/mapper/vg_kalkin-lv_home: UUID="6134733c-158e-4d62-a74d-e7d00e2ed8fd" TYPE="ext4" /dev/sdg1: LABEL="KINGSTON" UUID="6CE6-14FA" TYPE="vfat" [root@kalkin work]# mount /dev/sdg1 /mnt/test1 [root@kalkin work]# ls -1 /mnt/test1 [root@kalkin work]# cp hello.txt /mnt/test1/hello.txt [root@kalkin work]# ls -1 /mnt/test1 hello.txt [root@kalkin work]# cat /mnt/test1/hello.txt hello world [root@kalkin work]# umount /mnt/test1 I note that the Kingston memory stick UUID has changed from 7195-020C to 6CE6-14FA. Remove memory stick from Kalkin. Insert it into Aineko. It mounts automatically and an icon appears on the Desktop. On Aineko: aineko:~ stjohnpiano$ cd /Volumes/KINGSTON/ aineko:KINGSTON stjohnpiano$ ls hello.txt aineko:KINGSTON stjohnpiano$ cat hello.txt Mac aineko:KINGSTON stjohnpiano$ ls -lh total 32 -rwxrwxrwx 1 stjohnpiano staff 12B Jan 24 2018 hello.txt Strange. Download display_bytes.py (one of the downloadable assets included in article Listing_available_tools_on_Centos_6.9_Minimal edgecase 30 Listing available tools on Centos 6.9 Minimal ) to my Downloads directory. Copy hello.txt from the Kingston memory stick to my Downloads directory. Edit this value of the file_path variable in display_bytes.py so that the script targets hello.txt. file_path = "hello.txt" aineko:Downloads stjohnpiano$ python display_bytes.py [ord_0][ord_5][ord_22][ord_7][ord_0][ord_2][ord_0][ord_0]Mac Note: There is a space at the end of the output line. Looking at my ASCII reference table, I find that: - byte 0 is NUL (null) - byte 5 is ENQ (enquiry) - byte 22 is SYN (synchronous idle) - byte 7 is BEL (bell) - byte 2 is STX (start of text) I think at this point I can conclude that the problem is with the Kingston memory stick, not with Kalkin or the usage of the umount command. Solution: Put the Kingston memory stick into the disks-to-be-disposed-of pile. Tidy up: - delete display_bytes.py and hello.txt from Downloads directory.