大数据平台搭建(Ambari +HDP)

时间:2021-06-23 16:52:56   收藏:0   阅读:0

安装包

Ambari

HDP

HDP-UTILS

准备工作

集群节点规划

主机名 IP 功能 内存 磁盘
yum.hdp 192.168.229.30 Ambari/HDP packages 1G 50G
node-01.hdp 192.168.229.31 Ambari Server/Agent 3G 50G
node-02.hdp 192.168.229.32 Ambari Agent 2G 50G
node-03.hdp 192.168.229.33 Ambari Agent 2G 50G

克隆 4 台虚拟机

技术图片

修改主机名
# vi /etc/hostname
修改 IP 地址
# vi /etc/sysconfig/network-scripts/ifcfg-ens33
# service network restart
# ping www.baidu.com
hosts 文件添加主机名和IP地址映射

hosts 文件路径:C:\Windows\System32\drivers\etc

192.168.229.30 yum.hdp
192.168.229.31 node-01.hdp
192.168.229.32 node-02.hdp
192.168.229.33 ndoe-03.hpd
[root@node-01 ~]# vi /etc/hosts
192.168.229.30 yum.hdp
192.168.229.31 node-01.hdp
192.168.229.32 node-02.hdp
192.168.229.33 node-03.hdp
配置免密
[root@yum ~]# ssh-keygen -t rsa
[root@yum ~]# ssh-copy-id yum.hdp
[root@yum ~]# ssh-copy-id node-01.hdp
[root@yum ~]# ssh-copy-id node-02.hdp
[root@yum ~]# ssh-copy-id node-03.hdp

[root@node-01 ~]# ssh-keygen -t rsa
[root@node-01 ~]# ssh-copy-id node-01.hdp
[root@node-01 ~]# ssh-copy-id node-02.hdp
[root@node-01 ~]# ssh-copy-id node-03.hdp

[root@node-02 ~]# ssh-keygen -t rsa
[root@node-02 ~]# ssh-copy-id node-02.hdp
[root@node-02 ~]# ssh-copy-id node-03.hdp
安装 mysql

在 node-01.hdp 主机节点上安装 mysql,安装教程

在 yum.hdp 上安装 Ambari

安装 yum 相关工具

[root@yum ~]# yum install yum-utils -y
[root@yum ~]# yum repolist
[root@yum ~]# yum install createrepo -y

上传安装包到 yum.hdp

创建 hdp 目录,将安装包上传到该目录下

[root@yum ~]# mkdir hdp

技术图片

安装 Apache httpd

httpd 为 Apache HTTP 服务器程序,直接执行程序可启动服务器的服务

[root@yum ~]# yum install httpd -y

技术图片

技术图片

技术图片

配置本地 Repo 数据源

[HDP-GPL-3.1.0.0]
name=HDP-GPL Version - HDP-GPL-3.1.0.0
baseurl=http://yum.hdp/hdp-gpl/HDP-GPL/centos7
gpgcheck=1
gpgkey=http://yum.hdp/hdp-gpl/HDP-GPL/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1


- 分发 ambari.repo 和 hdp.repo 配置文件

```bash
[root@yum yum]# cd /etc/yum.repos.d
[root@yum yum]# scp ambari.repo node-01.hdp:$PWD
[root@yum yum]# scp ambari.repo node-02.hdp:$PWD
[root@yum yum]# scp ambari.repo node-03.hdp:$PWD
[root@yum yum]# scp hdp.repo node-01.hdp:$PWD
[root@yum yum]# scp hdp.repo node-02.hdp:$PWD
[root@yum yum]# scp hdp.repo node-03.hdp:$PWD

在 node-01.hdp 节点上 mysql 上创建相应的用户和数据库

在 node-01.hdp 节点上安装 Ambari-Server

[root@node-01 ~]# yum install ambari-server
[root@node-01 mysql]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is ‘enabled‘
SELinux mode is ‘enforcing‘
Temporarily disabling SELinux
WARNING: SELinux is set to ‘permissive‘ mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):root
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /root/apps/jdk1.8.0_141
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): node-01.hdp
Port (3306): 3306
Database name (ambari): ambari
Username (root): ambari
Enter Database Password (root):
Re-enter password:
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server ‘setup‘ completed successfully.
[root@node-01 mysql]# mysql -uambari -p
Enter password: #密码为Ambari123
mysql> use ambari;
Database changed
#执行 sql 脚本创建ambari数据库表
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
#查看数据库表是否创建成功
mysql> show tables;

启动Ambari-Server 服务

[root@node-01 mysql]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start........................................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server ‘start‘ completed successfully.

停止 Ambari-Server 服务: # ambari-server stop

安装Ambari-Agent

[root@node-01 ~]# yum -y install ambari-agent
[root@node-02 ~]# yum -y install ambari-agent
[root@node-03 ~]# yum -y install ambari-agent

访问Ambari web页面

默认端口8080,Username:admin;Password:admin;http://192.168.229.31:8080

技术图片

开始安装集群

技术图片

技术图片

技术图片

YUM 本地源如下:

HDP-3.1:http://yum.hdp/hdp/HDP/centos7

HDP-3.1-GPL:http://yum.hdp/hdp-gpl/HDP-GPL/centos7

HDP-UTILS-1.1.0.22:http://yum.hdp/hdp-utils/HDP-UTILS/centos7

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

#重新执行命令
[root@node-01 ~]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
Using python  /usr/bin/python
Setup ambari-server
Copying /usr/share/java/mysql-connector-java.jar to /var/lib/ambari-server/resources/mysql-connector-java.jar
If you are updating existing jdbc driver jar for mysql with mysql-connector-java.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server ‘setup‘ completed successfully.

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

技术图片

评论(0)
© 2014 mamicode.com 版权所有 京ICP备13008772号-2  联系我们:gaon5@hotmail.com
迷上了代码!