Timeline Service v2 默认集成嵌入HBase(HMaster、HRegionServer),进程启动 User: yarn-ats
- 嵌入HBase配置文件路径:
[yarn-ats@master1 embedded-yarn-ats-hbase]$ pwd /usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase [yarn-ats@master1 embedded-yarn-ats-hbase]$ ls -la total 40 drwxr-xr-x 2 yarn-ats hadoop 4096 3月 9 10:32 . drwxr-xr-x 4 root root 4096 4月 23 19:32 .. -rw-r--r-- 1 yarn-ats hadoop 1462 3月 7 09:22 hadoop-metrics2-hbase.properties -rw-r--r-- 1 yarn-ats hadoop 3374 4月 23 20:41 hbase-env.sh -rw-r--r-- 1 yarn-ats hadoop 818 3月 7 09:22 hbase_grant_permissions.sh -rw-r--r-- 1 yarn-ats hadoop 411 3月 7 09:22 hbase-policy.xml -rw-r--r-- 1 yarn-ats hadoop 5394 3月 7 09:22 hbase-site.xml -rw-r--r-- 1 yarn-ats hadoop 5315 3月 7 09:22 log4j.properties
-
hbase-site.xml文件
<configuration xmlns:xi="http://www.w3.org/2001/XInclude"> <property> <name>dfs.domain.socket.path</name> <value>/var/lib/hadoop-hdfs/dn_socket</value> </property> <property> <name>hbase.bucketcache.ioengine</name> <value></value> </property> <property> <name>hbase.bucketcache.percentage.in.combinedcache</name> <value></value> </property> <property> <name>hbase.bucketcache.size</name> <value></value> </property> <property> <name>hbase.client.keyvalue.maxsize</name> <value>1048576</value> </property> <property> <name>hbase.client.retries.number</name> <value>7</value> </property> <property> <name>hbase.client.scanner.caching</name> <value>100</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.coprocessor.master.classes</name> <value></value> </property> <property> <name>hbase.coprocessor.region.classes</name> <value></value> </property> <property> <name>hbase.coprocessor.regionserver.classes</name> <value></value> </property> <property> <name>hbase.defaults.for.version.skip</name> <value>true</value> </property> <property> <name>hbase.hregion.majorcompaction</name> <value>604800000</value> </property> <property> <name>hbase.hregion.majorcompaction.jitter</name> <value>0.50</value> </property> <property> <name>hbase.hregion.max.filesize</name> <value>10737418240</value> </property> <property> <name>hbase.hregion.memstore.block.multiplier</name> <value>4</value> </property> <property> <name>hbase.hregion.memstore.flush.size</name> <value>134217728</value> </property> <property> <name>hbase.hregion.memstore.mslab.enabled</name> <value>true</value> </property> <property> <name>hbase.hstore.blockingStoreFiles</name> <value>10</value> </property> <property> <name>hbase.hstore.compaction.max</name> <value>10</value> </property> <property> <name>hbase.hstore.compactionThreshold</name> <value>3</value> </property> <property> <name>hbase.local.dir</name> <value>${hbase.tmp.dir}/local</value> </property> <property> <name>hbase.master.info.bindAddress</name> <value>0.0.0.0</value> </property> <property> <name>hbase.master.info.port</name> <value>17010</value> </property> <property> <name>hbase.master.namespace.init.timeout</name> <value>2400000</value> </property> <property> <name>hbase.master.port</name> <value>17000</value> </property> <property> <name>hbase.master.ui.readonly</name> <value>false</value> </property> <property> <name>hbase.master.wait.on.regionservers.timeout</name> <value>30000</value> </property> <property> <name>hbase.regionserver.executor.openregion.threads</name> <value>20</value> </property> <property> <name>hbase.regionserver.global.memstore.size</name> <value>0.4</value> </property> <property> <name>hbase.regionserver.handler.count</name> <value>30</value> </property> <property> <name>hbase.regionserver.info.port</name> <value>17030</value> </property> <property> <name>hbase.regionserver.port</name> <value>17020</value> </property> <property> <name>hbase.rootdir</name> <value>/atsv2/hbase/data</value> </property> <property> <name>hbase.rpc.protection</name> <value>authentication</value> </property> <property> <name>hbase.rpc.timeout</name> <value>90000</value> </property> <property> <name>hbase.security.authentication</name> <value>simple</value> </property> <property> <name>hbase.security.authorization</name> <value>false</value> </property> <property> <name>hbase.superuser</name> <value>yarn</value> </property> <property> <name>hbase.tmp.dir</name> <value>/tmp/hbase-${user.name}</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>manager.bigdata.com,master2.bigdata.com,master1.bigdata.com</value> </property> <property> <name>hbase.zookeeper.useMulti</name> <value>true</value> </property> <property> <name>hfile.block.cache.size</name> <value>0.4</value> </property> <property> <name>zookeeper.recovery.retry</name> <value>6</value> </property> <property> <name>zookeeper.session.timeout</name> <value>90000</value> </property> <property> <name>zookeeper.znode.parent</name> <value>/atsv2-hbase-unsecure</value> </property> </configuration>
-
Zookeeper Cluster znode信息
[zookeeper@master1 zookeeper-client]$ bin/zkCli.sh -server master1.bigdata.com:2181 [zk: master1.bigdata.com:2181(CONNECTED) 1] ls /atsv2-hbase-unsecure [replication, meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, switch, master, running, draining, namespace, hbaseid, table] [zk: master1.bigdata.com:2181(CONNECTED) 2] ls /atsv2-hbase-unsecure/rs
- UI Interface
http://master1.bigdata.com:17010/
- Table list
prod.timelineservice.app_flow prod.timelineservice.application prod.timelineservice.entity prod.timelineservice.flowactivity prod.timelineservice.flowrun prod.timelineservice.subapplication
TTL: 默认30天 - Hbase shell执行
hbase --config /usr/hdp/3.1.0.0-78/hadoop-yarn/conf/embedded-yarn-ats-hbase shell hbase(main):004:0* status 1 active master, 0 backup masters, 1 servers, 0 dead, 124.0000 average load Took 1.2255 seconds hbase(main):005:0> list TABLE prod.timelineservice.app_flow prod.timelineservice.application prod.timelineservice.entity prod.timelineservice.flowactivity prod.timelineservice.flowrun prod.timelineservice.subapplication 6 row(s) Took 0.0862 seconds => ["prod.timelineservice.app_flow", "prod.timelineservice.application", "prod.timelineservice.entity", "prod.timelineservice.flowactivity", "prod.timelineservice.flowrun", "prod.timelineservice.subapplication"] hbase(main):018:0> scan 'prod.timelineservice.app_flow', { COLUMNS => ['m'], LIMIT => 10 }
- HDFS存储路径
/atsv2/hbase
原创文章,作者:kepupublish,如若转载,请注明出处:https://blog.ytso.com/193276.html