2021-01-08 20:20发布
以常用的编程语言php为例:
phpStudy v8.0 (针对Windows系统,免费)
针对Windows系统,一键安装,可以自行选择软件版本,你可在本地或者服务器端搭建与配置PHP运行环境。
主要功能:
1. 全新界面,更美观,操作更清晰
2. 安装包内置最新版本Apache、Nginx、MySQL、FTP、PHP
3. 在不同站点可以选择不同版本php互不影响(PHP多版本共存)
Linux系统服务器运行环境,肯定少不了最简单的宝塔面板(win也有对应的版本)。
安装要求:
内存:512M以上,推荐768M以上(纯面板约占系统60M内存)
硬盘:100M以上可用硬盘空间(纯面板约占20M磁盘空间)
系统:CentOS 7.1+ (Ubuntu16.04+.、Debian9.0+),确保是干净的操作系统,没有安装过其它环境带的Apache/Nginx/php/MySQL(已有环境不可安装)
Centos安装脚本: yum install -y wget && wget -O install.sh http://download.bt.cn/install/install_6.0.sh && sh install.sh
当然win和linux下的常用工具还不仅仅有这么多,比如Windows下的 xampp,wampserver等,linux下还有lnmp集成环境,OneinStack环境等
(1)禁用hugepage使用命令cat /sys/kernel/mm/transparent_hugepage/defrag查看是不是打开,如果不是nerver,需要配置。执行命令:runlevel,确定运行级别,ubuntu默认为2添加文件:/etc/init.d/dishuge
#!/bin/sh -e
echo "[dancylog]enable transparent_hugepage"
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
设置可执行权限:chmod 777 dishuge进入/etc/rc2.d目录(rc加运行级别),添加一个软连接:ln -s ../init.d/dishuge S10dishuge重启就可以了,可以使用命令:cat /sys/kernel/mm/transparent_hugepage/defrag确定是否设置成功了。(2)禁用swapecho "vm.swappiness=0" >> /etc/sysctl.confsysctl -p重启使用命令:cat /proc/sys/vm/swappines查看是否设置成功。(3)安装ssh,mysql,ntp
sudo apt-get install openssh-client=1:6.6p1-2ubuntu1
sudo apt-get install openssh-server
sudo apt-get install mysql-server mysql-client libmysql-java ntp
(4)卸载iptablessudo apt-get remove iptables(5)安装jdksudo mkdir /usr/lib/jvm进入jdk所在的目录sudo tar zxvf jdk-7u79-linux-x64.tar.gz -C /usr/lib/jvmsudo gedit /etc/profile
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
/etc/environment
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$JAVA_HOME/bin"
配置jdk默认版本
sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_79/bin/java 300
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_79/bin/javac 300
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/jdk1.7.0_79/bin/jar 300
sudo update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/jdk1.7.0_79/bin/javah 300
sudo update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/jdk1.7.0_79/bin/javap 300
sudo update-alternatives --config java
java -versionecho $JAVA_HOME(6)修改hostname需要修改2个配置文件/etc/hosts/etc/hostname根据需要更改就行了。(7)ssh免交互ssh-keygen -t rsacp id_rsa.pub authorized_keys不用更改权限,生成的文件每个电脑需要拷贝一份。(8)mysql 配置需要修改配置文件:/etc/mysql/my.cnf
bind-address = 0.0.0.0
//////////////////////下面的复制。
transaction-isolation = READ-COMMITTED
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
log_bin=/var/lib/mysql/mysql_binary_log
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 52M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES
重启mysqlservice mysql restart添加必要的数据库:
create database hive DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database am DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
grant all on hive.* to 'hl'@'%' identified by 'hl123';
grant all on hue.* to 'hl'@'%' identified by 'hl123';
grant all on am.* to 'hl'@'%' identified by 'hl123';
grant all on oozie.* to 'hl'@'%' identified by 'hl123';
grant all on scm.* to 'hl'@'%' identified by 'hl123';
FLUSH PRIVILEGES;
exit;
如果想删除对某个数据库的权限:revoke all on wifi.* from 'hl'@'%';(9)添加用户useradd --system --home=/opt/cm-5.4.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm(10) cm-5.4.0修改配置更改:/opt/cm-5.4.0/etc/cloudera-scm-agent/config.iniserver_host改为hostname中的名字。如果是agent,只需要更改config.ini,下面的不需要配置数据库/opt/cm-5.4.0/share/cmf/schema/scm_prepare_database.sh mysql scm -hlocalhost -uroot -p --scm-host localhost scm scm scm查看/opt/cm-5.4.0/etc/cloudera-scm-server/db.properties是否修改正确,如果不正确安装下面更改:
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=cloudera
com.cloudera.cmf.db.user=root
com.cloudera.cmf.db.password=hl123
(11)拷贝CDH5mkdir -p /opt/cloudera/parcel-repo/拷贝CDH5到这个目录下。(12)添加必要的目录mkdir -p /opt/cm-5.4.0/run/cloudera-scm-agent;mkdir -p /var/lib/cloudera-scm-server(13)启动server和agent/opt/cm-5.4.0/etc/init.d/cloudera-scm-server start/opt/cm-5.4.0/etc/init.d/cloudera-scm-agent start(14)卸载CDH5和重装删除数据库:
drop database scm;
drop database hive;
drop database hue;
drop database oozie;
drop database am;
删除用户:
userdel -r hdfs
userdel -r solr
userdel -r zookeeper
userdel -r llama
userdel -r httpfs
userdel -r mapred
userdel -r sqoop
userdel -r yarn
userdel -r kms
userdel -r hive
userdel -r sqoop2
userdel -r oozie
userdel -r hbase
userdel -r sentry
userdel -r impala
userdel -r spark
userdel -r hue
userdel -r cloudera-scm
groupdel hadoop
groupdel flume
groupdel hdfs
groupdel sqoop
groupdel hive
删除文件:
rm /impala/ /dfs/ /yarn/ -rf
rm /var/log/hive/ /var/log/hue/ /var/log/impalad/ /var/log/oozie/ /var/log/sqoop2/ /var/log/zookeeper/ -rf
rm /var/log/cloudera-scm-* /var/log/hadoop-* -rf
rm /var/tmp -rf
rm /var/lib/cloudera-* /var/lib/hadoop-* /var/lib/hue/ /var/lib/impala/ /var/lib/oozie/ /var/lib/sqoop2/ /var/lib/zookeeper/ /var/lib/spark/ -rf
需要
生产环境下应该如何搭配hadoop生态系统个组件版本查了很多资料,觉得不完善的话请在评论区补充,谢谢!:hadoopCHDhadoop生态系统生产环境版本搭配单机伪分布式Hadoop用于本机练习的话,hadoop版本自定义选择都可以,各个组件搭配也随意,也可以自己本机测试...
一、Impala简介Cloudera Impala对你存储在Apache Hadoop在HDFS,HBase的数据提供直接查询互动的SQL。除了像Hive使用相同的统一存储平台,Impala也使用相同的元数据,SQL语法(Hive SQL),ODBC驱动程序和用户界面(Hue Beeswax)。Impala还提供了一个熟悉的面...
catalog节点yum install -y impala-server impala impala-state-store impala-catalogworker 节点yum install -y impala-server impala配置打开 短路读取 和 打开块位置跟踪所谓的短路读取,就是允许impala把一些信息存储在本地磁盘上,可以加快计算的速度。...
Impala是Cloudera公司主导开发的新型查询系统,它提供SQL语义,能查询存储在Hadoop的HDFS和HBase中的PB级大数据。
最多设置5个标签!
以常用的编程语言php为例:
window系统推荐:
phpStudy v8.0 (针对Windows系统,免费)
针对Windows系统,一键安装,可以自行选择软件版本,你可在本地或者服务器端搭建与配置PHP运行环境。
主要功能:
1. 全新界面,更美观,操作更清晰
2. 安装包内置最新版本Apache、Nginx、MySQL、FTP、PHP
3. 在不同站点可以选择不同版本php互不影响(PHP多版本共存)
Linux系统服务器
Linux系统服务器运行环境,肯定少不了最简单的宝塔面板(win也有对应的版本)。
安装要求:
内存:512M以上,推荐768M以上(纯面板约占系统60M内存)
硬盘:100M以上可用硬盘空间(纯面板约占20M磁盘空间)
系统:CentOS 7.1+ (Ubuntu16.04+.、Debian9.0+),确保是干净的操作系统,没有安装过其它环境带的Apache/Nginx/php/MySQL(已有环境不可安装)
当然win和linux下的常用工具还不仅仅有这么多,比如Windows下的 xampp,wampserver等,linux下还有lnmp集成环境,OneinStack环境等
(1)禁用hugepage
使用命令cat /sys/kernel/mm/transparent_hugepage/defrag查看是不是打开,如果不是nerver,需要配置。
执行命令:runlevel,确定运行级别,ubuntu默认为2
添加文件:/etc/init.d/dishuge
#!/bin/sh -e
echo "[dancylog]enable transparent_hugepage"
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
设置可执行权限:chmod 777 dishuge
进入/etc/rc2.d目录(rc加运行级别),添加一个软连接:
ln -s ../init.d/dishuge S10dishuge
重启就可以了,可以使用命令:
cat /sys/kernel/mm/transparent_hugepage/defrag确定是否设置成功了。
(2)禁用swap
echo "vm.swappiness=0" >> /etc/sysctl.conf
sysctl -p
重启使用命令:
cat /proc/sys/vm/swappines
查看是否设置成功。
(3)安装ssh,mysql,ntp
sudo apt-get install openssh-client=1:6.6p1-2ubuntu1
sudo apt-get install openssh-server
sudo apt-get install mysql-server mysql-client libmysql-java ntp
(4)卸载iptables
sudo apt-get remove iptables
(5)安装jdk
sudo mkdir /usr/lib/jvm
进入jdk所在的目录
sudo tar zxvf jdk-7u79-linux-x64.tar.gz -C /usr/lib/jvm
sudo gedit /etc/profile
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
/etc/environment
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$JAVA_HOME/bin"
配置jdk默认版本
sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_79/bin/java 300
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_79/bin/javac 300
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/jdk1.7.0_79/bin/jar 300
sudo update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/jdk1.7.0_79/bin/javah 300
sudo update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/jdk1.7.0_79/bin/javap 300
sudo update-alternatives --config java
java -version
echo $JAVA_HOME
(6)修改hostname
需要修改2个配置文件
/etc/hosts
/etc/hostname
根据需要更改就行了。
(7)ssh免交互
ssh-keygen -t rsa
cp id_rsa.pub authorized_keys
不用更改权限,生成的文件每个电脑需要拷贝一份。
(8)mysql 配置
需要修改配置文件:/etc/mysql/my.cnf
bind-address = 0.0.0.0
//////////////////////下面的复制。
transaction-isolation = READ-COMMITTED
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
log_bin=/var/lib/mysql/mysql_binary_log
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 52M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES
重启mysql
service mysql restart
添加必要的数据库:
create database hive DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database am DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
grant all on hive.* to 'hl'@'%' identified by 'hl123';
grant all on hue.* to 'hl'@'%' identified by 'hl123';
grant all on am.* to 'hl'@'%' identified by 'hl123';
grant all on oozie.* to 'hl'@'%' identified by 'hl123';
grant all on scm.* to 'hl'@'%' identified by 'hl123';
FLUSH PRIVILEGES;
exit;
如果想删除对某个数据库的权限:revoke all on wifi.* from 'hl'@'%';
(9)添加用户
useradd --system --home=/opt/cm-5.4.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
(10) cm-5.4.0修改配置
更改:/opt/cm-5.4.0/etc/cloudera-scm-agent/config.ini
server_host改为hostname中的名字。
如果是agent,只需要更改config.ini,下面的不需要
配置数据库
/opt/cm-5.4.0/share/cmf/schema/scm_prepare_database.sh mysql scm -hlocalhost -uroot -p --scm-host localhost scm scm scm
查看/opt/cm-5.4.0/etc/cloudera-scm-server/db.properties是否修改正确,
如果不正确安装下面更改:
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=cloudera
com.cloudera.cmf.db.user=root
com.cloudera.cmf.db.password=hl123
(11)拷贝CDH5
mkdir -p /opt/cloudera/parcel-repo/
拷贝CDH5到这个目录下。
(12)添加必要的目录
mkdir -p /opt/cm-5.4.0/run/cloudera-scm-agent;mkdir -p /var/lib/cloudera-scm-server
(13)启动server和agent
/opt/cm-5.4.0/etc/init.d/cloudera-scm-server start
/opt/cm-5.4.0/etc/init.d/cloudera-scm-agent start
(14)卸载CDH5和重装
删除数据库:
drop database scm;
drop database hive;
drop database hue;
drop database oozie;
drop database am;
删除用户:
userdel -r hdfs
userdel -r solr
userdel -r zookeeper
userdel -r llama
userdel -r httpfs
userdel -r mapred
userdel -r sqoop
userdel -r yarn
userdel -r kms
userdel -r hive
userdel -r sqoop2
userdel -r oozie
userdel -r hbase
userdel -r sentry
userdel -r impala
userdel -r spark
userdel -r hue
userdel -r cloudera-scm
groupdel hadoop
groupdel flume
groupdel hdfs
groupdel sqoop
groupdel hive
删除文件:
rm /impala/ /dfs/ /yarn/ -rf
rm /var/log/hive/ /var/log/hue/ /var/log/impalad/ /var/log/oozie/ /var/log/sqoop2/ /var/log/zookeeper/ -rf
rm /var/log/cloudera-scm-* /var/log/hadoop-* -rf
rm /var/tmp -rf
rm /var/lib/cloudera-* /var/lib/hadoop-* /var/lib/hue/ /var/lib/impala/ /var/lib/oozie/ /var/lib/sqoop2/ /var/lib/zookeeper/ /var/lib/spark/ -rf
(1)禁用hugepage
使用命令cat /sys/kernel/mm/transparent_hugepage/defrag查看是不是打开,如果不是nerver,需要配置。
执行命令:runlevel,确定运行级别,ubuntu默认为2
添加文件:/etc/init.d/dishuge
#!/bin/sh -e
echo "[dancylog]enable transparent_hugepage"
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
设置可执行权限:chmod 777 dishuge
进入/etc/rc2.d目录(rc加运行级别),添加一个软连接:
ln -s ../init.d/dishuge S10dishuge
重启就可以了,可以使用命令:
cat /sys/kernel/mm/transparent_hugepage/defrag确定是否设置成功了。
(2)禁用swap
echo "vm.swappiness=0" >> /etc/sysctl.conf
sysctl -p
重启使用命令:
cat /proc/sys/vm/swappines
查看是否设置成功。
(3)安装ssh,mysql,ntp
sudo apt-get install openssh-client=1:6.6p1-2ubuntu1
sudo apt-get install openssh-server
sudo apt-get install mysql-server mysql-client libmysql-java ntp
(4)卸载iptables
sudo apt-get remove iptables
(5)安装jdk
sudo mkdir /usr/lib/jvm
进入jdk所在的目录
sudo tar zxvf jdk-7u79-linux-x64.tar.gz -C /usr/lib/jvm
sudo gedit /etc/profile
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
/etc/environment
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$JAVA_HOME/bin"
配置jdk默认版本
sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_79/bin/java 300
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_79/bin/javac 300
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/jdk1.7.0_79/bin/jar 300
sudo update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/jdk1.7.0_79/bin/javah 300
sudo update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/jdk1.7.0_79/bin/javap 300
sudo update-alternatives --config java
java -version
echo $JAVA_HOME
(6)修改hostname
需要修改2个配置文件
/etc/hosts
/etc/hostname
根据需要更改就行了。
(7)ssh免交互
ssh-keygen -t rsa
cp id_rsa.pub authorized_keys
不用更改权限,生成的文件每个电脑需要拷贝一份。
(8)mysql 配置
需要修改配置文件:/etc/mysql/my.cnf
bind-address = 0.0.0.0
//////////////////////下面的复制。
transaction-isolation = READ-COMMITTED
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
log_bin=/var/lib/mysql/mysql_binary_log
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 52M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES
重启mysql
service mysql restart
添加必要的数据库:
create database hive DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database am DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
grant all on hive.* to 'hl'@'%' identified by 'hl123';
grant all on hue.* to 'hl'@'%' identified by 'hl123';
grant all on am.* to 'hl'@'%' identified by 'hl123';
grant all on oozie.* to 'hl'@'%' identified by 'hl123';
grant all on scm.* to 'hl'@'%' identified by 'hl123';
FLUSH PRIVILEGES;
exit;
如果想删除对某个数据库的权限:revoke all on wifi.* from 'hl'@'%';
(9)添加用户
useradd --system --home=/opt/cm-5.4.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
(10) cm-5.4.0修改配置
更改:/opt/cm-5.4.0/etc/cloudera-scm-agent/config.ini
server_host改为hostname中的名字。
如果是agent,只需要更改config.ini,下面的不需要
配置数据库
/opt/cm-5.4.0/share/cmf/schema/scm_prepare_database.sh mysql scm -hlocalhost -uroot -p --scm-host localhost scm scm scm
查看/opt/cm-5.4.0/etc/cloudera-scm-server/db.properties是否修改正确,
如果不正确安装下面更改:
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=cloudera
com.cloudera.cmf.db.user=root
com.cloudera.cmf.db.password=hl123
(11)拷贝CDH5
mkdir -p /opt/cloudera/parcel-repo/
拷贝CDH5到这个目录下。
(12)添加必要的目录
mkdir -p /opt/cm-5.4.0/run/cloudera-scm-agent;mkdir -p /var/lib/cloudera-scm-server
(13)启动server和agent
/opt/cm-5.4.0/etc/init.d/cloudera-scm-server start
/opt/cm-5.4.0/etc/init.d/cloudera-scm-agent start
(14)卸载CDH5和重装
删除数据库:
drop database scm;
drop database hive;
drop database hue;
drop database oozie;
drop database am;
删除用户:
userdel -r hdfs
userdel -r solr
userdel -r zookeeper
userdel -r llama
userdel -r httpfs
userdel -r mapred
userdel -r sqoop
userdel -r yarn
userdel -r kms
userdel -r hive
userdel -r sqoop2
userdel -r oozie
userdel -r hbase
userdel -r sentry
userdel -r impala
userdel -r spark
userdel -r hue
userdel -r cloudera-scm
groupdel hadoop
groupdel flume
groupdel hdfs
groupdel sqoop
groupdel hive
删除文件:
rm /impala/ /dfs/ /yarn/ -rf
rm /var/log/hive/ /var/log/hue/ /var/log/impalad/ /var/log/oozie/ /var/log/sqoop2/ /var/log/zookeeper/ -rf
rm /var/log/cloudera-scm-* /var/log/hadoop-* -rf
rm /var/tmp -rf
rm /var/lib/cloudera-* /var/lib/hadoop-* /var/lib/hue/ /var/lib/impala/ /var/lib/oozie/ /var/lib/sqoop2/ /var/lib/zookeeper/ /var/lib/spark/ -rf
内存:512M以上,推荐768M以上(纯面板约占系统60M内存)
硬盘:100M以上可用硬盘空间(纯面板约占20M磁盘空间)
内存:512M以上,推荐768M以上(纯面板约占系统60M内存)
硬盘:100M以上可用硬盘空间(纯面板约占20M磁盘空间)
相关问题推荐
需要
生产环境下应该如何搭配hadoop生态系统个组件版本查了很多资料,觉得不完善的话请在评论区补充,谢谢!:hadoopCHDhadoop生态系统生产环境版本搭配单机伪分布式Hadoop用于本机练习的话,hadoop版本自定义选择都可以,各个组件搭配也随意,也可以自己本机测试...
一、Impala简介Cloudera Impala对你存储在Apache Hadoop在HDFS,HBase的数据提供直接查询互动的SQL。除了像Hive使用相同的统一存储平台,Impala也使用相同的元数据,SQL语法(Hive SQL),ODBC驱动程序和用户界面(Hue Beeswax)。Impala还提供了一个熟悉的面...
catalog节点yum install -y impala-server impala impala-state-store impala-catalogworker 节点yum install -y impala-server impala配置打开 短路读取 和 打开块位置跟踪所谓的短路读取,就是允许impala把一些信息存储在本地磁盘上,可以加快计算的速度。...
Impala是Cloudera公司主导开发的新型查询系统,它提供SQL语义,能查询存储在Hadoop的HDFS和HBase中的PB级大数据。