Hive安装过程(mysql/oracle存储元数据)
前置条件:
– mysql数据库已经安装成功
– hadoop环境已经配置正确,且可以提供正常服务
说明:
-由于资源有限,下面配置的hadoop集群只有一个节点,所有服务都在一个节点上启动
(一)mysql存储元数据
(1)创建mysql用户
[email protected]:~$ mysql -uroot -pmysql
mysql> CREATE USER 'hive' IDENTIFIED BY 'mysql';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%' WITH GRANT OPTION;
mysql> flush privileges;
(2)重启mysql服务
sudo service mysql restart
(3)建立 Hive 专用的元数据库(使用刚才创建的hive用户登陆)
mysql> exit;
hadoop@ubuntu:~$ mysql -uhive -pmysql
mysql> create database hive;
(4)Hive环境变量及配置项
- 设置环境变量
在/home/hadoop/.bashrc文件中加入环境变量
export HIVE_HOME=/home/hadoop/apache-hive-2.1.0-bin
export PATH=$HIVE_HOME/bin:$PATH
- 修改配置文件
将 $HIVE_HOME/conf/hive-default.xml.template修改为$HIVE_HOME/conf/hive-site.xml,修改hive-site.xml文件内容如下:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>Username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>mysql</value>
<description>password to use against metastore database</description>
</property>
(5)下载mysqlJDBC驱动包,复制到Hive的lib目录下
cp mysql-connector-java-5.1.39-bin.jar $HIVE_HOME/lib/
(6)初始化hive元数据表
schematool -initSchema -dbType mysql
(7)启动 Hive Shell
启动 Hive Shell, 执行“show tables;”命令,如果不报错,表明基于独立元数据库的 Hive 已经安装成功了
(8)测试
- 创建表
hive> create table mytable(str STRING);
- 查看表信息
hive> describe mytable;
OK
str string
Time taken: 0.075 seconds, Fetched: 1 row(s)
- 添加数据
echo "test hive table row" >/tmp/myfile
hive -e "LOAD DATA LOCAL INPATH '/tmp/myfile' INTO TABLE mytable" ;
- 查询数据
hive> select * from mytable;
OK
test hive table row
Time taken: 0.152 seconds, Fetched: 1 row(s)
- 删除表
hive> drop table mytable;
OK
Time taken: 2.133 seconds
(二)oracle 存储元数据
(1)创建用户
sqlplus "sys as sysdba"
SQL> create user hive identified by hive;
User created.
SQL> grant connect to hive;
Grant succeeded.
SQL> grant all privileges to hive;
Grant succeeded.
(2)配置文件
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:oracle:thin:@myhost:1521:xe</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>oracle.jdbc.OracleDriver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
(3)下载oracle驱动包
(4)初始化表
schematool -dbType oracle -initSchema
(5)示例(使用Hive自带的数据)
- 创建表
hive> CREATE TABLE testtable (foo INT, bar STRING) PARTITIONED BY (ds STRING);
- 插入数据
hive> LOAD DATA LOCAL INPATH '$HIVE_HOME/examples/files/kv1.txt' OVERWRITE INTO TABLE testtable PARTITION (ds='2016-07-28');
hive> LOAD DATA LOCAL INPATH '$HIVE_HOME/examples/files/kv2.txt' OVERWRITE INTO TABLE testtable PARTITION (ds='2016-07-29');
请将$HIVE_HOME替换成具体路径执行
– 查询数据
hive> select t.foo from testtable t where t.ds='2016-07-29';
(6)参考
http://www.cloudera.com/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_18_4.html
https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cdh_ig_hive_metastore_configure.html
原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/9574.html