spring hadoop 访问hbase入门详解大数据

1、  环境准备:

Maven

Eclipse

Java

Spring 版本 3..2.9

2、 Maven  pom.xml配置

<!-- Spring hadoop  --> 
				   <dependency> 
	        <groupId>org.apache.hbase</groupId> 
	        <artifactId>hbase-client</artifactId> 
	        <version>0.96.1.1-hadoop2</version> 
   		 </dependency> 
		<dependency> 
			<groupId>org.springframework.data</groupId> 
			<artifactId>spring-data-jpa</artifactId> 
			<version>1.6.0.RELEASE</version> 
		</dependency> 
		 <dependency> 
	        <groupId>org.springframework.data</groupId> 
	        <artifactId>spring-data-hadoop</artifactId> 
	        <version>2.0.2.RELEASE</version> 
   		 </dependency>

3、 Spring和hadoop、hbase相关配置文件

  <?xml version="1.0" encoding="UTF-8"?> 
<beans xmlns="http://www.springframework.org/schema/beans" 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc" 
	xmlns:tx="http://www.springframework.org/schema/tx" xmlns:util="http://www.springframework.org/schema/util" 
	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context" 
	xmlns:mongo="http://www.springframework.org/schema/data/mongo" 
	xmlns:hdp="http://www.springframework.org/schema/hadoop" 
	xmlns:cache="http://www.springframework.org/schema/cache" xmlns:c="http://www.springframework.org/schema/c" 
	xmlns:p="http://www.springframework.org/schema/p" 
	xsi:schemaLocation="   
        http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd   
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd   
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd   
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd   
        http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-mongo.xsd 
        http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd   
        http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd    
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"> 

  其中标红的是spring  hadoop xml命名空间配置。

Hadoop hbase相关配置文件如下:

   <!–  默认properties–>

    <hdp:configuration>fs.default.name=hdfs://192.98.8.224:8010</hdp:configuration>

    <hdp:hbase-configuration delete-connection=“${delete-connection}” zk-quorum=“${hbase.zookeeper.quorum}” zk-          port=“${hbase.zookeeper.property.clientPort}”/>

对应的properties如下:

hbase.zookeeper.property.clientPort=2181

hbase.zookeeper.quorum=192.98.8.224

hbase.master=192.98.8.224:600000

fs.default.name=hdfs://192.98.8.224:8010

delete-connection=true

#hive jdbc url

hive.url=jdbc:hive://192.98.8.224:10000/default

spring hbasetemplate配置如下:

    <bean id=“hbaseTemplate” class=“org.springframework.data.hadoop.hbase.HbaseTemplate”> <property name=“configuration” ref=“hbaseConfiguration” />

</bean>

Hbasetemplate使用代码示例:

Tile t = hbaseTemplate.get("GW_TILES", "0_1_1", new RowMapper<Tile>() { 
 
			@Override 
			public Tile mapRow(Result result, int rowNum) throws Exception { 
				// TODO Auto-generated method stub 
				 
				Tile t = new Tile(); 
				t.setData(result.getValue("T".getBytes(), "key".getBytes())); 
				return t; 
			} 
		}); 

  

Hbasetemplate 常用方法简介:

      hbaseTemplate.get(“GW_TILES”, “0_1_1”, new RowMapper  常用于查询,使用示例如下所示:

Tile t = hbaseTemplate.get("GW_TILES", "0_1_1", new RowMapper<Tile>() { 
 
			@Override 
			public Tile mapRow(Result result, int rowNum) throws Exception { 
				// TODO Auto-generated method stub 
				 
				Tile t = new Tile(); 
				t.setData(result.getValue("T".getBytes(), "key".getBytes())); 
				return t; 
			} 
		}); 

  hbaseTemplate.execute(dataIdentifier, new TableCallback 常用于更新操作,使用示例如下所示:

return hbaseTemplate.execute(dataIdentifier, new TableCallback<Boolean>() { 
 
			@Override 
			public Boolean doInTable(HTableInterface table) throws Throwable { 
				// TODO Auto-generated method stub 
				boolean flag = false; 
				try{ 
				Delete delete = new Delete(key.getBytes()); 
				table.delete(delete); 
				flag = true; 
				}catch(Exception e){ 
					e.printStackTrace(); 
				} 
				return flag; 
			} 
		}); 

  

备注:spring hbasetemplate针对hbase接口做了强大的封装,普通功能可以使用它强大的接口,同时复杂的功能,还可以使用hbase原生的接口,如:HTableInterface、Result等。其类方法如下图:

spring hadoop 访问hbase入门详解大数据

同时hbasetemplate封装了hbase连接池等,它的创建和释放通过配置来自动管理。

 

文章转载请注明出处:http://www.cnblogs.com/likehua/p/4016257.html

 

原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/9584.html

(0)
上一篇 2021年7月19日
下一篇 2021年7月19日

相关推荐

发表回复

登录后才能评论