一、環境準備

1、服務器資源分配

(資料圖片)

2、禁用selinux

> vi /etc/selinux/config SELINUX=disabled

3、禁用防火墻

> systemctl disable firewalld> systemctl stop firewalld

4、安裝JDK

> tar -zxvf jdk-8u102-linux-x64.tar.gz -C /usr/local/> vi /etc/profile.d/java.shexport JAVA_HOME=/usr/local/jdk1.8.0_102 export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$JAVA_HOME:$PATH> source /etc/profile

二、部署Kerberos

1、安裝Kerberos，通過在線安裝的方式安裝KDC客戶端

> yum -y install krb5-server krb5-libs krb5-workstation

2、配置krb5.conf

> vi /etc/krb5.conf[libdefaults]default_realm = NIFI.COM  #此處需要進行配置[realms]NIFI.COM ={      kdc = server   #此處配置的為主機名server    admin_server = server  #同上, 配置的為主機名server}[domain_realm].nifi.com = NIFI.COM  #此處配置與前面配置[realms]保持一致nifi.com = NIFI.COM

Realm參數自己設定但要一致kdc=主機ipadmin_server=主機ipKdc為kdc位置，格式為 主機：端口（可用默認）（主機可以是主機名或ip）

> scp /etc/krb5.conf 192.168.0.12:/etc/> scp /etc/krb5.conf 192.168.0.13:/etc/

3、配置kdc.conf

> vi /var/kerberos/krb5kdc/kdc.conf

[realms] NIFI.COM = {   #master_key_type = aes256-cts   acl_file = /var/kerberos/krb5kdc/kadm5.acl   dict_file = /usr/share/dict/words   admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab   supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal }此處為NIFI.COM與/etc/krb5.conf中的配置保持一致

4、配置kadm5.acl

> vi /var/kerberos/krb5kdc/kadm5.acl */admin@NIFI.COM *

5、創建Kerberos數據庫

> kdb5_util create -r NIFI.COM -sEnter KDC database master key: 123456a?Re-enter KDC database master key to verify: 123456a?

當長時間Loading random data時，另開一個控制臺使用如下命令

> cat /dev/sda > /dev/urandom

6、添加用戶

> kadmin.localkadmin.local: addprinc superadminEnter password for principal "admin/admin@NIFI.COM": superadminRe-enter password for principal "admin/admin@NIFI.COM":superadminkadmin.local: addprinc testEnter password for principal "admin/admin@NIFI.COM": 123456a?Re-enter password for principal "admin/admin@NIFI.COM":123456a?kadmin.local: exit

7、啟動Kerberos

> systemctl start krb5kdc> systemctl start kadmin> systemctl enable krb5kdc> systemctl enable kadmin

8、 添加服務主體并導出keytab

> kadmin.local> addprinc -randkey test/NIFI> ktadd -k /opt/test-NIFI.keytab test/NIFI> q

9、創建web登陸用戶創建login/server@NIFI.COM作為管理員賬戶，自定義密碼為hadoop

kadmin.local -q "addprinc login/server"

使用kinit命令來檢測創建的用戶是否成功(需要輸入密碼)

kinit login/server@NIFI.COM

三、證書生成

利用Toolkit工具生成證書，Nifi官方提供了一個工具用戶生成相關的證書，參考文檔 Apache NiFi Admin Guide 的 TLS Generation Toolkit模塊:1、下載與安裝下載地址：http://nifi.apache.org/download.html下載后解壓安裝nifi-toolkit-1.16.0-bin.zip (或tar.gz格式，自己選擇最新版本)，配置好環境變量方便使用。2、生成相關證書為server節點生成密鑰庫，truststore，nifi.properties以及具有給定DN的客戶端證書。

bin/tls-toolkit.sh standalone -O -n "172.16.0.11" -C "CN=superadmin, OU=NIFI" --subjectAlternativeNames "172.16.0.11,0.0.0.0" -o "target1"

其中-c, -certificateAuthorityHostname 為Hostname of NiFi Certificate Authority。-C,–clientCertDn 生成適合在指定DN的瀏覽器中使用的客戶端證書（可以指定多次）-n，-hostnames要生成證書的主機名列表(以逗號分隔)，可指定多次，支持范圍和實例模式。-o, -outputDirectory 指定生成證書的路徑-O,–isOverwrite 覆蓋現有主機輸出–subjectAlternativeNames 在證書中用作主題備用名稱的域的逗號分隔列表3、分發證書

把node1目錄下的文件拷貝到node1節點下nifi的安裝目錄的conf文件下，重復的文件選擇覆蓋，同樣把node2目錄下的文件拷貝到node2節點下nifi的安裝目錄的conf文件下，把node3目錄下的文件拷貝到node3節點下nifi的安裝目錄的conf文件下。

scp ./192.168.0.11/* /usr/local/nifi-1.16.0/conf/scp ./192.168.0.12/* root@192.168.0.12:/usr/local/nifi-1.16.0/conf/scp ./192.168.0.13/* root@192.168.0.13:/usr/local/nifi-1.16.0/conf/

四、NIFI配置

1、 配置nifi常用參數采用nifi-toolkit工具生成的證書，在生成證書的過程中已經生了一個nifi.properties配置文件，把不同目錄下的證書文件拷貝到不同節點nifi的conf目錄下，重復的文件選擇覆蓋然后修改nifi.properties中的一些參數

nifi.state.management.embedded.zookeeper.start=truenifi.cluster.protocol.is.secure=truenifi.cluster.is.node=true nifi.zookeeper.connect.string=192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181

采用了nifi中的自帶zookeeper，設置三個節點的zookeeper.修改配置了zookeeper節點的nifi安裝目錄下的/conf/state-management.xml

192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181

修改每個節點的/conf/zookeeper.properties

server.1=192.168.0.11:2888:3888;2181server.2=192.168.0.12:2888:3888;2181server.3=192.168.0.13:2888:3888;2181

創建myid文件,主要根據zookeeper.properties文件的的dataDir=./state/zookeeper參數確定位置在此文件夾下（沒有則創建）不同節點分別執行

echo 1 > ./state/zookeeper/myidecho 2 > ./state/zookeeper/myidecho 3 > ./state/zookeeper/myid

2、配置kerberos參數

vi nifi.propertiesnifi.remote.input.secure=truenifi.security.user.login.identity.provider=kerberos-providernifi.security.user.authorizer=file-providernifi.sensitive.props.key=123456789012 #所有選項都需要至少 12 個字符的密碼（nifi.sensitive.props.key值）# kerberos # nifi.kerberos.krb5.file=/etc/krb5.confnifi.kerberos.service.principal=test/NIFI@NIFI.COM #配置的服務主體nifi.kerberos.service.keytab.location=/opt/test-NIFI.keytab #配置服務主體的keytab

把配置服務主體的/opt/test-NIFI.keytab拷貝到每個節點的/opt目錄下

scp test-NIFI.keytab root@192.168.0.12:/opt/scp test-NIFI.keytab root@192.168.0.13:/opt/

3、 配置nifi登陸信息

vi login-identity-providers.xml

   kerberos-provider   org.apache.nifi.kerberos.KerberosProvider   NIFI.COM   12 hours 

放開kerberos-provide的配置，把相關注釋部分取消，然后配置成你前面設置的Realm。4、配置用于登陸的用戶信息

vi authorizers.xml

        file-provider        org.apache.nifi.authorization.FileAuthorizer        ./conf/authorizations.xml        ./conf/users.xml                superadmin@NIFI.COM                       CN=192.168.0.11, OU=NIFI        CN=192.168.0.12, OU=NIFI        CN=192.168.0.13, OU=NIFI 

5、配置JVM修改 bootstrap.conf，默認2048太小

> java.arg.2=-Xms10240m> java.arg.3=-Xmx10240m

6、 啟動服務，登陸web頁面配置完成后，每個節點啟動nifi：nifi.sh start，啟動完成后用jps查看相關進程，或用curl命令來驗證nifi是否啟動：curl --insecure https://192.168.0.11:9443/nifi每個節點都能登陸訪問web頁面

Hosts:設置的ip

Port: 9443(默認，或者你修改的端口)

User: superadmin

Password: ****(前面kerberos配置的用戶名和密碼)

五、集群搭建過程中出現的問題

1、2019-04-08 08:26:28,522 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/current-user to x.x.x.x:9443 due to

解決辦法： 使用 tls-toolkit 重新創建證書，需要指定subjectAlternativeNames

2019-04-08 08:26:28,522 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/current-user to x.x.x.x:9443 due to javax.net.ssl.SSLPeerUnverifiedException: Hostname x.x.x.x not verified: certificate: sha256/IiBnANAzEOL~~~qm7wcLzM= DN: CN=x.x.x.x, OU=NIFI subjectAltNames: [x.x.x.x]2019-04-08 08:26:28,523 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicatorjavax.net.ssl.SSLPeerUnverifiedException: Hostname x.x.x.x not verified:certificate: sha256/IiBnANAzEOLVg+DipeiOT+fkIDpqtA3h/Nhqm7wcLzM=DN: CN=x.x.x.x, OU=NIFIsubjectAltNames: [x.x.x.x]at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:316)at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270)at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162)at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)at okhttp3.RealCall.execute(RealCall.java:77)at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:138)at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:132)at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647)at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)

2、當查看或刪除隊列數據時出現下面錯誤解決辦法：需要對每個節點添加數據讀和寫的權限

permission error: Insufficient Permissions Node nifi-test-1:8443 is unable to fulfill this request due to: Unable to view the data for Processor with ID b3169c5d-0175-1000-0000-00001734159f. Contact the system administrator. Contact the system administrator.

六、頁面展示

登陸頁面：

登陸成功：

集群管理頁面