Read Mounting HDFS. There is something very interesting. As people know HDFS is Hadoop Distributed File System. So, i wanted to test a bit about hadoop-fuse-dfs. Somehow, I had impala for test only. I thought it's easy to use it and learn.
-bash-4.1$ dfFrom above example, I mount hdfs to /mnt path and modified file. After saved file (wq), I used "cat" command but no data in file. Then tried again, i found data. :)
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 24607156 4370100 19987060 18% /
devtmpfs 1534680 200 1534480 1% /dev
tmpfs 1544584 112 1544472 1% /dev/shm
/dev/sda1 24607156 4370100 19987060 18% /
-bash-4.1$ sudo hadoop-fuse-dfs rw -oserver=127.0.0.1 /mnt/
INFO /data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.1.0/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_options.c:164 Adding FUSE arg /mnt/
-bash-4.1$ df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 24607156 4370104 19987056 18% /
devtmpfs 1534680 200 1534480 1% /dev
tmpfs 1544584 112 1544472 1% /dev/shm
/dev/sda1 24607156 4370104 19987056 18% /
fuse_dfs 24576000 0 24576000 0% /mnt
-bash-4.1$
-bash-4.1$ ls /mnt/user/cloudera/
file.out file.txt
-bash-4.1$ cat /mnt/user/cloudera/file.txt
1,Surachart
2,John
3,Jame
-bash-4.1$
-bash-4.1$ hadoop fs -cat file.txt
1,Surachart
2,John
3,Jame
-bash-4.1$ cat /mnt/user/cloudera/file.txt
1,Surachart
2,John
3,Jame
-bash-4.1$ vi /mnt/user/cloudera/file.txt
-bash-4.1$
-bash-4.1$ cat /mnt/user/cloudera/file.txt
-bash-4.1$-- no data, used "cat" command again --
-bash-4.1$ cat /mnt/user/cloudera/file.txt
1,Surachart
2,John
3,Jame
4,Chart
-bash-4.1$ hadoop fs -cat file.txt
1,Surachart
2,John
3,Jame
4,Chart
-bash-4.1$ hadoop fs -ls
Found 4 items
-rw-r--r-- 1 root cloudera 0 2012-11-01 09:46 .file.txt.swp
-rw-r--r-- 1 root cloudera 0 2012-11-01 09:49 .file.txt.swx
drwxr-xr-x - cloudera cloudera 0 2012-11-01 09:29 file.out
-rw-r--r-- 1 cloudera cloudera 34 2012-11-01 09:58 file.txt
Then, tested creating file.
-bash-4.1$ hadoop fs -ls file2.txt
ls: `file2.txt': No such file or directory
-bash-4.1$ echo "TEST1" > /mnt/user/cloudera/file2.txt
-bash-4.1$ echo "TEST2" >> /mnt/user/cloudera/file2.txt
-bash-4.1$ cat /mnt/user/cloudera/file2.txt
TEST1
TEST2
-bash-4.1$ hadoop fs -ls file2.txt
Found 1 items
-rw-r--r-- 1 cloudera cloudera 12 2012-11-01 11:08 file2.txt
-bash-4.1$ hadoop fs -cat file2.txt
TEST1
TEST2
However, there supports some... operations.
Supports reads, writes, and directory operations (e.g., cp, ls, more, cat, find, less, rm, mkdir, mv, rmdir). Things like touch, chmod, chown, and permissions are in the works. Fuse-dfs currently shows all files as owned by nobody.
If tested more... Maybe, you might find something weird(not sure why?).
-bash-4.1$ cat /mnt/user/cloudera/file2.txt
TEST1
TEST2
-bash-4.1$ echo "TEST" > /mnt/user/cloudera/file2.txt
-bash-4.1$ cat /mnt/user/cloudera/file2.txt
-bash-4.1$-- no data --
-bash-4.1$ hadoop fs -cat file2.txt
-bash-4.1$-- no data --
Maybe, someone can help to explain :)... So, just learn and test before use.
No comments:
Post a Comment