Presto详细安装和测试--单节点和集群模式
参考:
https://prestodb.github.io/docs/current/installation/deployment.html
https://www.imooc.com/video/19187
presto server下载地址:https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.226/
presto client下载地址:https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.226/
环境:
presto-server-0.226(要求java版本151+)
jdk1.8.0_221
单节点模式:
coordinator和work在同一个节点
etc/config.properties
If you are setting up a single machine for testing that will function as both a coordinator and worker, use this configuration:
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=9999
query.max-memory=50GB
query.max-memory-per-node=5GB
query.max-total-memory-per-node=10GB
discovery-server.enabled=true
discovery.uri=http://192.168.0.9:9999
注意:如果query.max-memory-per-node配置的过小,会在查询的时候导致OOM,导致presto server服务对应的worker挂掉
etc/node.properties
node.environment=production
node.id=presto1
node.data-dir=/data/presto
node.environment需要保证每个节点相同
node.id需要保证每个节点的唯一性
node.data-dir为presto运行日志目录,运行日志在${node.data-dir}/var/log目录下
etc/jvm.config
-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
etc/log.properties
com.facebook.presto=INFO
etc/catalog/hive.properties
参考:https://prestodb.github.io/docs/current/connector/hive.html#hive-configuration-properties
hive.metastore.uri为hive的metastore对应的thift地址,
可以查看hive的hive-site.xml配置文件查询相应配置。
connector.name=hive-hadoop2
hive.metastore.uri=thrift://192.168.0.5:9083
hive.config.resources=${HADOOP_HOME}/etc/hadoop/conf/core-site.xml,${HADOOP_HOME}/etc/hadoop/conf/hdfs-site.xml
配置完成后,启动./bin/launcher start
通过jps,
可以看到PrestoServer已经启动, 如果未启动,可以查看presto的日志,查找原因
验证:
可以看到有一个worker启动了
测试:
启动客户端:
下载相应版本的客户端jar(https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.226/)
./presto-cli-0.226-executable.jar --server 192.168.0.9:9999 --catalog hive --schema default
schema为对应的数据库实例
2.集群模式
集群模式唯一的区别在于worker节点的etc/config.properties
配置如下
coordinator=false
http-server.http.port=9999
query.max-memory=50GB
query.max-memory-per-node=5GB
query.max-total-memory-per-node=10GB
discovery.uri=http://192.168.0.9:9999
启动worker节点
./launcher start
可以看到workers已经变为2了
更多推荐
所有评论(0)