一、说明:
1 2 3 4 5 |
常用网址: 官网地址:http://opentsdb.net/ 下载地址:https://github.com/OpenTSDB/opentsdb/releases 官方说明文档:http://opentsdb.net/docs/build/html/index.html |
> opentsdb需要zookeeper和hbase的支持,请提前安装和启动好它们,才能正常启动和使用opentsdb。
> opentsdb安装使用也需要提前配置好java环境,请确认环境变量配置测试好javac命令是否能正常使用。
在使用opentsdb之前,为了确保能连接到zookeeper,可以telnet其端口,然后执行stats
命令
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
[root@imzcy ~]# telnet 127.0.0.1 2181 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. stats Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT Clients: /127.0.0.1:39746[1](queued=0,recved=1908,sent=1908) /127.0.0.1:39876[0](queued=0,recved=1,sent=0) /127.0.0.1:39738[1](queued=0,recved=1953,sent=1954) /127.0.0.1:39744[1](queued=0,recved=1863,sent=1863) /127.0.0.1:39740[1](queued=0,recved=1864,sent=1864) /127.0.0.1:39742[1](queued=0,recved=1865,sent=1865) /127.0.0.1:39736[1](queued=0,recved=4028,sent=4065) Latency min/avg/max: 0/0/197 Received: 13537 Sent: 13574 Connections: 7 Outstanding: 0 Zxid: 0x99 Mode: standalone Node count: 44 Connection closed by foreign host. [root@imzcy ~]# |
二、安装部署openTSDB
0、首先安装opentsdb需要的依赖包
GnuPlot是一个命令行的交互式绘图工具,使用opentsdb查询数据时可以通过它来以曲线图直观的展现出来(不安装也不影响opentsdb使用)。
git就后面编译的时候会用到这个命令,没有的话会报错,我们也装上吧。
1 |
[root@imzcy ~]# yum -y install gnuplot git |
1、解压下载好的包
1 2 |
[root@imzcy ~]# tar zxvf opentsdb-2.3.0.tar.gz -C /opt/ [root@imzcy ~]# cd /opt/opentsdb-2.3.0/ |
2、编译配置(如果编译有报错请至本文最下方7.1和7.2处查找解决方法)
1 |
[root@imzcy opentsdb-2.3.0]# ./build.sh |
3、在hbase中创建opentsdb需要的表(需指定hbase的家目录)
1 |
[root@imzcy opentsdb-2.3.0]# env COMPRESSION=NONE HBASE_HOME=/opt/hbase-1.1.4/ ./src/create_table.sh |
> 其中COMPRESSION值可以是NONE,LZO,GZIP或SNAPPY。
> 如果您只是在评估OpenTSDB,那么现在不要担心压缩。在生产和大规模生产中,请确保使用有效的压缩库,因为它可以极大地节省存储空间。
4、复制src/opentsdb.conf配置文件到当前目录(tsdb命令默认调用opentsdb家目录下的配置文件,如果没有会报错),在末尾追加以下配置内容(不能将以下配置直接添加到配置文件行首,不然会不起作用)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@imzcy opentsdb-2.3.0]# cp src/opentsdb.conf ./ [root@imzcy opentsdb-2.3.0]# vi opentsdb.conf tsd.storage.compaction.flush_speed = 1 tsd.network.port = 4242 tsd.http.staticroot =build/staticroot tsd.http.cachedir = /tmp/tsd tsd.core.auto_create_metrics = true tsd.http.query.allow_delete = true tsd.http.request.enable_chunked = true tsd.http.request.max_chunk = 2097152 tsd.network.keep_alive = false tsd.storage.hbase.zk_quorum=127.0.0.1:2181 tsd.query.skip_unresolved_tagvs=true tsd.storage.fix_duplicates=true |
为了获得最佳性能,请确保你的临时目录使用tmpfs
tsd.http.cachedir - 临时文件写入的路径
tsd.http.staticroot - 指定静态GUI文件的路径./build/staticroot
tsd.storage.hbase.zk_quorum - 如果HBase和Zookeeper未在同一台计算机上运行,请在此处指定主机和端口。
opentsdb配置的缓存目录,通过内置的GUI要求的图形生成存储临时文件。应定期清除这些文件以释放空间。OpenTSDB此时不会自行清理,但是有一个脚本应该每天至少运行一次作为cron运行tools/clean_cache.sh。
5、根据当前家目录下的配置文件定义,来启动opentsdb
1 |
[root@imzcy opentsdb-2.3.0]# ./build/tsdb tsd |
在启动的时候也可以指定选项来覆盖配置文件中定义的内容
1 2 3 4 |
[root@imzcy opentsdb-2.3.0]# cat zcy.sh #!/bin/bash ./build/tsdb tsd --port=4243 --staticroot=build/staticroot --cachedir="/tmp/tsd" --zkquorum=127.0.0.1:2181 [root@imzcy opentsdb-2.3.0]# sh zcy.sh |
6、访问opentsdb
启动好之后,就可以通过浏览器打开http://ip:port进行访问。
graph:用来查询数据,以曲线图方式展现出现。输入起止时间和metric名称即可查询。
stats:显示当前metric
logs:显示当前opentsdb一些日志信息。
version:当前opentsdb版本信息
7、常见问题
7.1、使用./build.sh编译时报错:configure: error: cannot find javac
出现这个错误一般是java环境没有配置好,命令行输入java然后连敲两次Tab键看自动补齐是否有javac这个命令。如果没有可以尝试给$JAVA_HOME/bin目录下所有文件可执行权限或则重新配置java环境。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@imzcy opentsdb-2.3.0]# ./build.sh + test -f configure + test -d build + mkdir build + cd build + test -f Makefile + ../configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for md5sum... /usr/bin/md5sum checking for java... /opt/jdk1.8.0_121/bin/java checking for javac... no configure: error: cannot find javac [root@imzcy opentsdb-2.3.0]# |
7.2、使用./build.sh编译时报错:javac: 找不到文件: ./src/net/opentsdb/query/expression/parser/*.java
不知道什么原因,好像每安装opentsdb都会遇到这个问题,需要复制third_party目录下所有内容到build目录下,然后在重新执行./build.sh即可。
1 2 |
[root@imzcy opentsdb-2.3.0]# cp -r third_party/ build/ [root@imzcy opentsdb-2.3.0]# ./build.sh |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@imzcy opentsdb-2.3.0]# ./build.sh ...省略部分内容... /opt/jdk1.8.0_121/bin/java -cp third_party/javacc/javacc-6.1.2.jar javacc -OUTPUT_DIRECTORY:./src/net/opentsdb/query/expression/parser ../src/parser.jj; echo PWD: `pwd`; 错误: 找不到或无法加载主类 javacc PWD: /opt/opentsdb-2.3.0/build /opt/jdk1.8.0_121/bin/javac -Xlint -source 6 -encoding utf-8 -d . -cp ../third_party/jexl/commons-logging-1.1.1.jar:../third_party/guava/guava-18.0.jar:../third_party/slf4j/log4j-over-slf4j-1.7.7.jar:../third_party/logback/logback-classic-1.0.13.jar:../third_party/logback/logback-core-1.0.13.jar:../third_party/jackson/jackson-annotations-2.4.3.jar:../third_party/jackson/jackson-core-2.4.3.jar:../third_party/jackson/jackson-databind-2.4.3.jar:../third_party/javacc/javacc-6.1.2.jar:../third_party/jexl/commons-jexl-2.1.1.jar:../third_party/jgrapht/jgrapht-core-0.9.1.jar:../third_party/netty/netty-3.9.4.Final.jar:../third_party/slf4j/slf4j-api-1.7.7.jar:../third_party/suasync/async-1.4.0.jar:../third_party/apache/commons-math3-3.4.1.jar:../third_party/hbase/asynchbase-1.7.2.jar:../third_party/protobuf/protobuf-java-2.5.0.jar:../third_party/zookeeper/zookeeper-3.4.6.jar: ../src/core/AggregationIterator.java ../src/core/Aggregator.java ../src/core/Aggregators.java ../src/core/AppendDataPoints.java ../src/core/BatchedDataPoints.java ../src/core/ByteBufferList.java ../src/core/ColumnDatapointIterator.java ../src/core/CompactionQueue.java ../src/core/Const.java ../src/core/DataPoint.java ../src/core/DataPoints.java ../src/core/DataPointsIterator.java ../src/core/Downsampler.java ../src/core/DownsamplingSpecification.java ../src/core/FillingDownsampler.java ../src/core/FillPolicy.java ../src/core/IncomingDataPoint.java ../src/core/IncomingDataPoints.java ../src/core/IllegalDataException.java ../src/core/Internal.java ../src/core/MutableDataPoint.java ../src/core/Query.java ../src/core/QueryException.java ../src/core/RateOptions.java ../src/core/RateSpan.java ../src/core/RowKey.java ../src/core/RowSeq.java ../src/core/SaltScanner.java ../src/core/SeekableView.java ../src/core/Span.java ../src/core/SpanGroup.java ../src/core/TSDB.java ../src/core/Tags.java ../src/core/TsdbQuery.java ../src/core/TSQuery.java ../src/core/TSSubQuery.java ../src/core/WritableDataPoints.java ../src/core/WriteableDataPointFilterPlugin.java ../src/graph/Plot.java ../src/meta/Annotation.java ../src/meta/MetaDataCache.java ../src/meta/TSMeta.java ../src/meta/TSUIDQuery.java ../src/meta/UIDMeta.java ../src/query/QueryUtil.java ../src/query/expression/Absolute.java ../src/query/expression/Alias.java ../src/query/expression/DiffSeries.java ../src/query/expression/DivideSeries.java ../src/query/expression/EDPtoDPS.java ../src/query/expression/Expression.java ../src/query/expression/ExpressionDataPoint.java ../src/query/expression/ExpressionFactory.java ../src/query/expression/ExpressionIterator.java ../src/query/expression/ExpressionReader.java ../src/query/expression/Expressions.java ../src/query/expression/ExpressionTree.java ../src/query/expression/HighestCurrent.java ../src/query/expression/HighestMax.java ../src/query/expression/IntersectionIterator.java ../src/query/expression/ITimeSyncedIterator.java ../src/query/expression/NumericFillPolicy.java ../src/query/expression/MovingAverage.java ../src/query/expression/MultiplySeries.java ../src/query/expression/PostAggregatedDataPoints.java ../src/query/expression/Scale.java ../src/query/expression/SumSeries.java ../src/query/expression/TimeShift.java ../src/query/expression/TimeSyncedIterator.java ../src/query/expression/UnionIterator.java ../src/query/expression/VariableIterator.java ../src/query/filter/TagVFilter.java ../src/query/filter/TagVLiteralOrFilter.java ../src/query/filter/TagVNotKeyFilter.java ../src/query/filter/TagVNotLiteralOrFilter.java ../src/query/filter/TagVRegexFilter.java ../src/query/filter/TagVWildcardFilter.java ../src/query/pojo/Downsampler.java ../src/query/pojo/Expression.java ../src/query/pojo/Filter.java ../src/query/pojo/Join.java ../src/query/pojo/Metric.java ../src/query/pojo/Output.java ../src/query/pojo/Query.java ../src/query/pojo/Timespan.java ../src/query/pojo/Validatable.java ../src/search/SearchPlugin.java ../src/search/SearchQuery.java ../src/search/TimeSeriesLookup.java ../src/stats/Histogram.java ../src/stats/StatsCollector.java ../src/stats/QueryStats.java ../src/tools/ArgP.java ../src/tools/CliOptions.java ../src/tools/CliQuery.java ../src/tools/CliUtils.java ../src/tools/DumpSeries.java ../src/tools/Fsck.java ../src/tools/FsckOptions.java ../src/tools/MetaPurge.java ../src/tools/MetaSync.java ../src/tools/Search.java ../src/tools/StartupPlugin.java ../src/tools/TSDMain.java ../src/tools/TextImporter.java ../src/tools/TreeSync.java ../src/tools/UidManager.java ../src/tree/Branch.java ../src/tree/Leaf.java ../src/tree/Tree.java ../src/tree/TreeBuilder.java ../src/tree/TreeRule.java ../src/tsd/AbstractHttpQuery.java ../src/tsd/AnnotationRpc.java ../src/tsd/BadRequestException.java ../src/tsd/ConnectionManager.java ../src/tsd/DropCachesRpc.java ../src/tsd/GnuplotException.java ../src/tsd/GraphHandler.java ../src/tsd/HttpJsonSerializer.java ../src/tsd/HttpSerializer.java ../src/tsd/HttpQuery.java ../src/tsd/HttpRpc.java ../src/tsd/HttpRpcPlugin.java ../src/tsd/HttpRpcPluginQuery.java ../src/tsd/LineBasedFrameDecoder.java ../src/tsd/LogsRpc.java ../src/tsd/PipelineFactory.java ../src/tsd/PutDataPointRpc.java ../src/tsd/QueryExecutor.java ../src/tsd/QueryRpc.java ../src/tsd/RpcHandler.java ../src/tsd/RpcPlugin.java ../src/tsd/RpcManager.java ../src/tsd/RpcUtil.java ../src/tsd/RTPublisher.java ../src/tsd/SearchRpc.java ../src/tsd/StaticFileRpc.java ../src/tsd/StatsRpc.java ../src/tsd/StorageExceptionHandler.java ../src/tsd/SuggestRpc.java ../src/tsd/TelnetRpc.java ../src/tsd/TreeRpc.java ../src/tsd/UniqueIdRpc.java ../src/tsd/WordSplitter.java ../src/uid/FailedToAssignUniqueIdException.java ../src/uid/NoSuchUniqueId.java ../src/uid/NoSuchUniqueName.java ../src/uid/RandomUniqueId.java ../src/uid/UniqueId.java ../src/uid/UniqueIdFilterPlugin.java ../src/uid/UniqueIdInterface.java ../src/utils/ByteArrayPair.java ../src/utils/ByteSet.java ../src/utils/Config.java ../src/utils/DateTime.java ../src/utils/Exceptions.java ../src/utils/FileSystem.java ../src/utils/JSON.java ../src/utils/JSONException.java ../src/utils/Pair.java ../src/utils/PluginLoader.java ../src/utils/Threads.java ../src/tools/BuildData.java ./src/net/opentsdb/query/expression/parser/*.java javac: 找不到文件: ./src/net/opentsdb/query/expression/parser/*.java 用法: javac <options> <source files> -help 用于列出可能的选项 make[1]: *** [.javac-stamp] 错误 2 make[1]: 离开目录“/opt/opentsdb-2.3.0/build” make: *** [all] 错误 2 [root@imzcy opentsdb-2.3.0]# |
7.3、使用tsdb
命令来启动opentsdb的时候,报错:Config: No configuration found, will use defaults
Missing static root directory
出现这个问题一般是opentsdb的配置文件没有定义好或则配置文件没有放到opentsdb的家目录下面,导致启动时没有正确加载到配置。这个可详细参考上面第5步的配置!
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
[root@imzcy opentsdb-2.3.0]# ./build/tsdb tsd 2018-09-20 01:20:29,090 INFO [main] TSDMain: Starting. 2018-09-20 01:20:29,120 INFO [main] TSDMain: net.opentsdb.tools 2.3.0 built at revision cac608a (MINT) 2018-09-20 01:20:29,120 INFO [main] TSDMain: Built on 2016/12/29 13:57:15 +0000 by root@centos.localhost:/home/hobbes/opentsdb_OFFICIAL/build 2018-09-20 01:20:29,131 INFO [main] Config: No configuration found, will use defaults Missing static root directory Usage: tsd --port=PORT --staticroot=PATH --cachedir=PATH Starts the TSD, the Time Series Daemon --async-io=true|false Use async NIO (default true) or traditional blocking io --auto-metric Automatically add metrics to tsdb as they are inserted. Warning: this may cause unexpected metrics to be tracked --backlog=NUM Size of connection attempt queue (default: 3072 or kernel somaxconn. --bind=ADDR Address to bind to (default: 0.0.0.0). --cachedir=PATH Directory under which to cache result of requests. --config=PATH Path to a configuration file (default: Searches for file see docs). --disable-api=true|false Set tsd.core.enable_api to false (default true) --disable-ui=true|false Set tsd.core.enable_ui to false (default true) --flush-interval=MSEC Maximum time for which a new data point can be buffered (default: 1000). --max-connections=NUM Maximum number of connections to accept --port=NUM TCP port to listen on. --read-only=true|false Set tsd.mode to ro (default false) --staticroot=PATH Web root from which to serve static files (/s URLs). --statswport Force all stats to include the port --table=TABLE Name of the HBase table where to store the time series (default: tsdb). --uidtable=TABLE Name of the HBase table to use for Unique IDs (default: tsdb-uid). --worker-threads=NUM Number for async io workers (default: cpu * 2). --zkbasedir=PATH Path under which is the znode for the -ROOT- region (default: /hbase). --zkquorum=SPEC Specification of the ZooKeeper quorum to use (default: localhost). [root@imzcy opentsdb-2.3.0]# |