1增加主机映射(与namenode的映射一样)：
增加最后一行
[[email protected] ~]# su - root
1 [[email protected] ~]# vi /etc/hosts 2 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 3 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 4 192.168.48.129 hadoop-master 5 [[email protected] ~]#
2新建用户hadoop
建立hadoop用户组
新建用户，useradd -d /usr/hadoop -g hadoop -m hadoop （新建用户hadoop指定用户主目录/usr/hadoop 及所属组hadoop）
passwd hadoop 设置hadoop密码（这里设置密码为hadoop）
[[email protected] ~]# groupadd hadoop
[[email protected] ~]# useradd -d /usr/hadoop -g hadoop -m hadoop
[[email protected] ~]# passwd hadoop
3配置jdk环境
本次安装的是hadoop-2.7.5，需要JDK 7以上版本。若已安装可跳过。
JDK安装可参考：https://www.cnblogs.com/shihaiming/p/5809553.html
或者直接拷贝master上的JDK文件更有利于保持版本的一致性。
[[email protected] java]# su - root
[[email protected] java]# mkdir -p /usr/java
[[email protected] java]# scp -r [email protected]:/usr/java/jdk1.7.0_79 /usr/java
[[email protected] java]# ll
total 12
drwxr-xr-x. 8 root root 4096 Feb 13 01:34 default
drwxr-xr-x. 8 root root 4096 Feb 13 01:34 jdk1.7.0_79
drwxr-xr-x. 8 root root 4096 Feb 13 01:34 latest
设置Java及hadoop环境变量
确保/usr/java/jdk1.7.0.79存在
su – root
vi /etc/profile
确保/usr/java/jdk1.7.0.79存在
1 unset i 2 unset -f pathmunge 3 JAVA_HOME=/usr/java/jdk1.7.0_79 4 CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar 5 PATH=/usr/hadoop/hadoop-2.7.5/bin:$JAVA_HOME/bin:$PATH
设置生效（重要）
[[email protected] ~]# source /etc/profile [[email protected] ~]#
JDK安装后确认：
[[email protected] ~]$ java -version java version "1.7.0_79" Java(TM) SE Runtime Environment (build 1.7.0_79-b15) Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode) [[email protected] ~]$
4设置hadoop的环境变量
拷贝namenode上已配置好的hadoop目录到当前主机
[[email protected] ~]# su - hadoop Last login: Sat Feb 24 14:04:55 CST 2018 on pts/1 [[email protected] ~]$ pwd /usr/hadoop
[[email protected] ~]$ scp -r [email protected]:/usr/hadoop/hadoop-2.7.5 .
The authenticity of host 'hadoop-master (192.168.48.129)' can't be established.
ECDSA key fingerprint is 1e:cd:d1:3d:b0:5b:62:45:a3:63:df:c7:7a:0f:b8:7c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-master,192.168.48.129' (ECDSA) to the list of known hosts.
[email protected]'s password:
[[email protected] ~]$ ll total 0 drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Desktop drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Documents drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Downloads drwxr-xr-x 10 hadoop hadoop 150 Feb 24 14:30 hadoop-2.7.5 drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Music drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Pictures drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Public drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Templates drwxr-xr-x 2 hadoop hadoop 6 Feb 24 11:32 Videos [[email protected] ~]$
到此，Hadoop的客户端安装就算完成了，接下来就可以使用了。
执行hadoop命令结果如下，
[[email protected] ~]$ hadoop Usage: hadoop [--config confdir] [COMMAND | CLASSNAME] CLASSNAME run the class named CLASSNAME or where COMMAND is one of: fs run a generic filesystem user client version print the version jar <jar> run a jar file note: please use "yarn jar" to launch YARN applications, not this command. checknative [-a|-h] check native hadoop and compression libraries availability distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive classpath prints the class path needed to get the credential interact with credential providers Hadoop jar and the required libraries daemonlog get/set the log level for each daemon trace view and modify Hadoop tracing settings Most commands print help when invoked w/o parameters. [[email protected] ~]$
5.使用hadoop
创建本地文件
[[email protected] ~]$ hdfs dfs -ls
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2018-02-22 23:41 output
[[email protected] ~]$ vi my-local.txt hello boy! yehyeh
上传本地文件至集群
[[email protected] ~]$ hdfs dfs -mkdir upload [[email protected] ~]$ hdfs dfs -ls upload [[email protected] ~]$ hdfs dfs -ls Found 2 items drwxr-xr-x - hadoop supergroup 0 2018-02-22 23:41 output drwxr-xr-x - hadoop supergroup 0 2018-02-23 22:38 upload
[[email protected] ~]$ hdfs dfs -ls upload
[[email protected] ~]$ hdfs dfs -put my-local.txt upload
[[email protected] ~]$ hdfs dfs -ls upload
Found 1 items
-rw-r--r-- 3 hadoop supergroup 18 2018-02-23 22:45 upload/my-local.txt
[[email protected] ~]$ hdfs dfs -cat upload/my-local.txt
hello boy!
yehyeh
[[email protected] ~]$
ps:注意本地java版本与master拷贝过来的文件中/etc/hadoop-env.sh配置的JAVA_HOME是否要保持一致没有验证过，本文是保持一致的
原创文章，作者：奋斗，如若转载，请注明出处：https://blog.ytso.com/9428.html