Once your cluster is secured, you probably want to start allowing users to access the cluster and you may not want to issue individual certificates for each user. In this case, one of the option is to use LDAP as the authentication provider of NiFi. This is quite simple, and we’ll see in this post how to easily setup a local LDAP server and integrate NiFi with it.
In terms of configuration, everything is done with two files:
- ./conf/nifi.properties
- ./conf/login-identity-providers.xml
In nifi.properties, we are interested by two properties:
nifi.login.identity.provider.configuration.file
nifi.security.user.login.identity.provider
The first one is used to give the path to the login-identity-providers.xml and the second one is used to define the name of the identity provider to use from the XML file (in case you configured multiple providers).
A quick quote from the documentation:
NiFi supports user authentication via client certificates or via username/password. Username/password authentication is performed by a Login Identity Provider. The Login Identity Provider is a pluggable mechanism for authenticating users via their username/password. Which Login Identity Provider to use is configured in two properties in the nifi.properties file.
The
nifi.login.identity.provider.configuration.fileproperty specifies the configuration file for Login Identity Providers. Thenifi.security.user.login.identity.providerproperty indicates which of the configured Login Identity Provider should be used. If this property is not configured, NiFi will not support username/password authentication and will require client certificates for authenticating users over HTTPS. By default, this property is not configured meaning that username/password must be explicitly enabled.NiFi does not perform user authentication over HTTP. Using HTTP all users will be granted all roles.
In other words, if you want login/password authentication, your cluster needs to be secured first!
OK, so I set the following values in nifi.properties:
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.security.user.login.identity.provider=ldap-providerAnd then I just need to configure my XML files and to restart NiFi. Here are the LDAP parameters (and we can notice that the identifier is matching the value set in nifi.properties):
<provider> <identifier>ldap-provider</identifier> <class>org.apache.nifi.ldap.LdapProvider</class> <property name="Authentication Strategy">START_TLS</property> <property name="Manager DN"></property> <property name="Manager Password"></property> <property name="TLS - Keystore"></property> <property name="TLS - Keystore Password"></property> <property name="TLS - Keystore Type"></property> <property name="TLS - Truststore"></property> <property name="TLS - Truststore Password"></property> <property name="TLS - Truststore Type"></property> <property name="TLS - Client Auth"></property> <property name="TLS - Protocol"></property> <property name="TLS - Shutdown Gracefully"></property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url"></property> <property name="User Search Base"></property> <property name="User Search Filter"></property> <property name="Identity Strategy">USE_DN</property> <property name="Authentication Expiration">12 hours</property> </provider>
And here is the associated documentation:
Identity Provider for users logging in with username/password against an LDAP server.
‘Authentication Strategy’ – How the connection to the LDAP server is authenticated. Possible values are ANONYMOUS, SIMPLE, LDAPS, or START_TLS.
‘Manager DN’ – The DN of the manager that is used to bind to the LDAP server to search for users.
‘Manager Password’ – The password of the manager that is used to bind to the LDAP server to search for users.‘TLS – Keystore’ – Path to the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.
‘TLS – Keystore Password’ – Password for the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.
‘TLS – Keystore Type’ – Type of the Keystore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12).
‘TLS – Truststore’ – Path to the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.
‘TLS – Truststore Password’ – Password for the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.
‘TLS – Truststore Type’ – Type of the Truststore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12).
‘TLS – Client Auth’ – Client authentication policy when connecting to LDAP using LDAPS or START_TLS. Possible values are REQUIRED, WANT, NONE.
‘TLS – Protocol’ – Protocol to use when connecting to LDAP using LDAPS or START_TLS. (i.e. TLS, TLSv1.1, TLSv1.2, etc).
‘TLS – Shutdown Gracefully’ – Specifies whether the TLS should be shut down gracefully before the target context is closed. Defaults to false.‘Referral Strategy’ – Strategy for handling referrals. Possible values are FOLLOW, IGNORE, THROW.
‘Connect Timeout’ – Duration of connect timeout. (i.e. 10 secs).
‘Read Timeout’ – Duration of read timeout. (i.e. 10 secs).‘Url’ – Url of the LDAP server (i.e. ldap://<hostname>:<port>).
‘User Search Base’ – Base DN for searching for users (i.e. CN=Users,DC=example,DC=com).
‘User Search Filter’ – Filter for searching for users against the ‘User Search Base’. (i.e. sAMAccountName={0}). The user specified name is inserted into ‘{0}’.‘Identity Strategy’ – Strategy to identify users. Possible values are USE_DN and USE_USERNAME. The default functionality if this property is missing is USE_DN in order to retain backward compatibility. USE_DN will use the full DN of the user entry if possible. USE_USERNAME will use the username the user logged in with.
‘Authentication Expiration’ – The duration of how long the user authentication is valid for. If the user never logs out, they will be required to log back in following this duration.
OK, enough theory, let’s install a LDAP server using Apache Directory Studio. This project provides an easy way to setup a LDAP server but is also providing a great GUI to manage/administrate existing LDAP servers.I’ll go quick because it’s quite simple to setup and if needed the documentation of the official website is very useful.
Once downloaded and installed, just launch it. On the workbench, we are going to create a new server. Click on the ‘+’ symbol in the “LDAP Servers” tab:
Then, select Apache DS and give it a name:
Create a connection: right click on your server / create a connection. And start your server to access it. You should be able to access the Overview tab of your server. We are going to create a partition/branch for NiFi users:
Click on Advanced Partitions configuration and then Add a new partition. Here I decided to call my partition “dc=nifi,dc=com”:
At this point, you need to restart your server (right click / stop, right click / start).
Now we are going to create an organizational unit for groups and an organizational unit for people. In the ou=groups, we will define two groups, one for normal users and one for administrators. And we are going to create one user in each group, a user “test” in the group “users”, and a user “admin” in the group “admins”. This can be done through the GUI but in this case, I’ll do it by importing the below LDIF file:
dn: ou=people,dc=nifi,dc=com objectclass: organizationalUnit objectClass: extensibleObject objectclass: top ou: people dn: ou=groups,dc=nifi,dc=com objectclass: organizationalUnit objectClass: extensibleObject objectclass: top ou: groups dn: cn=users,ou=groups,dc=nifi,dc=com objectClass: groupOfUniqueNames objectClass: top cn: users uniqueMember: cn=test,ou=people,dc=nifi,dc=com dn: cn=admins,ou=groups,dc=nifi,dc=com objectClass: groupOfUniqueNames objectClass: top cn: admins uniqueMember: cn=admin,ou=people,dc=nifi,dc=com dn: cn=test,ou=people,dc=nifi,dc=com objectclass: inetOrgPerson objectclass: organizationalPerson objectclass: person objectclass: top cn: test description: A test user sn: test uid: test mail: test@nifi.com userpassword: password dn: cn=admin,ou=people,dc=nifi,dc=com objectclass: inetOrgPerson objectclass: organizationalPerson objectclass: person objectclass: top cn: admin description: A admin user sn: admin uid: admin mail: admin@nifi.com userpassword: password
To import it, right click on dc=nifi,dc=com, then Import, then LDIF import and select your file.
This will give you the following structure:
Now we want to configure NiFi to connect to our LDAP server. For that you have to note that, by default, the manager of the server (for an Apache DS LDAP server) has “uid=admin,ou=system” as DN and “secret” as password. Then the XML file is configured as below (no LDAPS/TLS in this example):
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<loginIdentityProviders>
<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">uid=admin,ou=system</property>
<property name="Manager Password">secret</property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://localhost:10389</property>
<property name="User Search Base">ou=people,dc=nifi,dc=com</property>
<property name="User Search Filter">uid={0}</property>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="Authentication Expiration">12 hours</property>
</provider>
</loginIdentityProviders>
We need to restart NiFi to take into account the modifications. Note: if NiFi is clustered, configuration files must be the same on all nodes.
Now… if you try to connect as test or admin, you will get the following error:
Unknown user with identity ‘admin’. Contact the system administrator.
This is because you first need to add this user in the list of users through NiFi UI using the initial admin account (see Apache NiFi 1.1.0 – Secured cluster setup). At there is no syncing mechanism to automatically add LDAP users/groups into NiFi.
When connected with your initial admin account (using your individual certificate), go into users to add your users, and then into policies to grant access and rights to the users:
You have now a NiFi instance integrated with a LDAP server and you can connect as different users defined in your LDAP. It gives you the opportunity to add users and play with the policy model implemented in NiFi.
Important note: NiFi has a large and active community, new features regarding LDAP integration could be provided very soon (for example: NIFI-3115).
As always, comments/remarks are welcomed!
Pierre Villard 
After configured the xml, I restart nifi. Then I get this Exception.
Caused by: org.apache.nifi.authentication.exception.ProviderCreationException: TLS – Protocol must be specified.
Can u tell me how can i start with no TLS-Protocol?
LikeLike
Would you mind sharing your LDAP configuration? Is your LDAP server secured?
LikeLike
Thanks a lot for your reply. It turns out that my LDAP configuration is wrong. Now I can login nifi sucessfully. But there is a new problem. I create a user named ‘admin’ and I can access NiFi web ui after logged in with ldap user ‘admin’.
It shows “Unable to perform the desired action due to insufficient permissions. Contact the system administrator.” in the page.
What should I do next?Sorry for asking too much, I am pretty new for nifi. Wish your reply
LikeLike
Basically, setting up LDAP is not enough, it allows you to authenticate against NiFi but, by default, all the users from your LDAP directory have no rights at all. You need to connect to NiFi as the “initial admin user” (using a client certificate – no password) and then to add the user in the NiFi users (NiFi menu / Users) and then to grant the permissions to this user (NiFi menu / Policies). Let me know if it does not make sense to you.
LikeLike
Thanks, I connect to NiFi as the “initial admin user”. Then a add a user named “admin” then a grant the permissions to admin. This is my “authorizations.xml” file
This is my “users.xml” file
nifi-user.log like this
Authentication success for cn=admin,ou=people,dc=nifi,dc=com
o.a.n.w.a.c.AccessDeniedExceptionMapper cn=admin,ou=people,dc=nifi,dc=com does not have permission to access the requested resource. Returning Forbidden response.
LikeLike
Once your user is added in Users, you should grant the permission “view the user interface” to this user in Policies. It’ll allow you to get on the NiFi canvas with your user (but nothing else, unless you grant the correct other permissions).
LikeLike
I granted the permission “view the user interface” to user “admin” in Policies menu. Then I restart nifi, but the problem still exists. nifi-user.log file:AccessDeniedExceptionMapper cn=admin,ou=people,dc=nifi,dc=com does not have permission to access the requested resource.
Any suggests? Thanks in advance.
LikeLike
Can you share authorizations.xml file, users.xml file in a gist (gist.github.com) online? Also what did you configure for “Identity Strategy” in your LDAP configuration? Can you try to add a user using the full DN instead of just admin and see if it’s better?
LikeLike
https://github.com/learnfromzero/nifi-authorizations
I commit my configuration into github. I create LDAP server by Apache Directory Studio that this article suggests. Where to configure “Identity Strategy”?
LikeLike
Identity Strategy is defined in the LDAP configuration (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldap_login_identity_provider). You can specify to use the username (based on the filter you defined) or the full DN. In your case, I’d try to use the full DN when adding a user in NiFi and see if it’s better.
LikeLike
I’m really sincerely grateful for all your help！ My problem is solved now. Wish you a have good day!
LikeLike
Great! Happy NiFi-ing!
LikeLike
After all changing on xml, I got this error below:
2017-06-12 20:53:19,245 INFO [NiFi Web Server-17] o.a.n.w.a.c.IllegalArgumentExceptionMapper java.lang.IllegalArgumentException: The supplied username and password are not valid.. Returning Bad Request response.
I already check all the configuration, but still work. Follow all the examples available on the internet, and documentation, but still having error on nifi-user.log.
Can u help me?
LikeLike
The best way to check your configuration is by running ldapsearch commands using the same parameters as you set in the XML configuration file.
LikeLike
Hello, can we configure single nifi with ldap (no sercure)
LikeLiked by 1 person
No. There is no point to enable authentication if NiFi is not secured and can be accessed over HTTP (login/password would go unencrypted over the wire).
LikeLike
Hello pvillard31 , i have got any error
creating bean with name ‘loginIdentityProvider’: FactoryBean threw exception on object creation; nested exception is java.lang.Exception: Unable to load the login identity provider configuration file at: /hxxx/bizxxxxxime/nifi-1.3.0/./conf/login-identity-providers.xml
[Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server…
o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector@caee26e{HTTP/1.1,[http/1.1]}{0.0.0.0:9090}
org.eclipse.jetty.server.session Stopped scavenging
Can i know wat reason
LikeLike
You should have much more logs / stack traces in NiFi log files. It looks like the configuration you set in the login identity provider file is not correct.
LikeLike
Hi pvillard31,
Could you please help me i am getting following error in nifi user log while accessing nifi using a client cert.ERROR : CN=kylo, OU=NIFI, O=NIFI, L=HYD, ST=TN, C=IN], groups[] does not have permission to access the requested resource. Unknown user with identity ‘CN=kylo, OU=NIFI, O=NIFI, L=HYD, ST=TN, C=IN’. Returning Forbidden response.
Below is my entry in authorisers.xml in intial admin
“CN=kylo, OU=NIFI, O=NIFI, L=HYD, ST=TN, C=IN”
Can you let me know what is wrong here.
LikeLike
Hi Sharique, please check what has been generated in the authorizations and users XML files after you restarted NiFi after you applied your changes. Note that you’d have to delete the two files if you update the configuration and expect changes there.
LikeLike
Hi,
I’m trying to integrate Nifi with Ldap. I made necessary changes as given above. Now nifi UI is working fine but I do not get the login page. Please suggest, where I’m missing the configuration.
—
ldap-provider
org.apache.nifi.ldap.LdapProvider
LDAPS
uid=nifiadmin,ou=nifi,dc=abc,dc=com
nifiadmin-password
/usr/hdf/current/nifi/conf/keystore.jks
changeit
JKS
/usr/hdf/current/nifi/conf/truststore.jks
changeit
JKS
REQUIRED
TLS
FOLLOW
10 secs
10 secs
ldaps://ldap-dallas.abc.com:8443
cn=nifiadmin,ou=nifi,dc=abc,dc=com
sAMAccountName={0}
12 hours
—
Also, How to verify that Nifi is successfully integrated with ldap or not?
LikeLike
Best is to try authenticating with a user from LDAP. Note that appropriate authorization need to be set for this user if you want to access the canvas. Authorizations can be set using the initial admin user.
LikeLike
Hi,
In addition to above query:
1. I do not see users tab in Nifi UI main menu.
2. By default it logins with anonymous user. I can see at the Log tab just beside to main menu.. After selecting login tab and putting credentials into that it gives error as given below:
“Unable to validate the supplied credentials. Please contact the system administrator.”
Complete configuration posted in
https://community.hortonworks.com/questions/139404/how-to-integrate-nifi-with-ldap-by-using-ranger-po.html
Note: Ranger is integrated with LDAP and able to login ranger UI trough domain users.
Please suggest if anything missing through.
LikeLike
NiFi has to be secured (HTTPS) to allow user authentication / authorization management.
LikeLike
Complete setup scenario:
In a cluster ( HDF 3.0.1 – Ambari, Nifi, zookeeper, Ranger, DB – Mysql ), all componants are running fine. Nifi UI is configured with HTTPS but do not get successful login page in Nifi UI.
(To configure Nifi UI with HTTPS – converted keystore.jks file into pks12 format and loaded the pks12 file into browser)
Ranger is integrated with LDAP successfully. Ranger UI is accessible through LDAP users.
Copied Nifi’s keystore and trustore file from Nifi server to Ranger server to build the trust between them. (copied at /usr/hdf/current/ranger-admin/conf) Then Ranger Policy is created and added LDAP users in it. Also given Read and Write permissions to added LDAP users in Ranger policy.
Now there is one issue. If I add some LDAP users in the Ranger policy then I cannot access Nifi UI. I got ‘insuffecient permissions and unable to access the page’ kind of errors. Logs shows authentication is success for LDAP users but authorization is failed.
But If I gave {users} in user’s section of Ranger Policy, then I can login Nifi UI with my LDAP user. Also Nifi UI can be accessible by anonymous user. I dont know from where anonymous user is coming.
But If I remove {user} from user section then I cannot login with LDAP user as well as anonymous user.
As per some blogs, I found it could be the related from authorizations.xml and users.xml files. But those files are missing from Nifi servers.
How to create/generate those files ?
Nifi Config
nifi.security.user.login.identity.provider
ldap-provider
Template for login-identity-providers.xml
ldap-provider
org.apache.nifi.ldap.LdapProvider
USE_USERNAME
SIMPLE
CN=zxc_oi,OU=fox,DC=abc,DC=com
xxx
FOLLOW
10 secs
10 secs
ldap://ldap.abc.com:389
DC=abc,DC=com
sAMAccountName={0}
12 hours
LikeLike
Hi,
I’d double check the policies definition in Ranger and look at the audit logs to see what is denied by Ranger.
LikeLike
Thank you @pvillard31
I found the solution. Issue is fixed now.
In my case, one of LDAP username is ‘dvteam’ but in LDAP database there was full description of username as ‘architecture dev team, locations, team details, etc’.
Error messages I found in nifi-user.log. is ‘architecture dev team’ user was trying to authenticate with nifi nodes. Authentication was successful but authorizations not happening.
The username which I’ve mentioned in initial admin identity was ‘dvteam’.(cn=dvteam,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com) Then as per logs, I changed it to (cn=architecture dev team,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com)
Also there was some mismatch about host names in node identities section. ‘hostname -f’ shows a hostname ip-zz-xx-ec2-internal. So, I have given ‘ip-zz-xx-ec2-internal’ in node identities section but that was not working. Then I have changed the hostnames to ‘nifi1.abc.local’ and mentioned in node identities.
In ‘Template for login-identity-providers.xml’ I’ve made some changes. Earlier I had set ‘use_username’ in ‘USE_DN’ this section.
later I’ve changed to use_dn. because as per nifi-user log authentication is happening with LDAP user ‘architecture dev team’.
So in my case user_username was not working for authentications.
Every configurations changes I used to remove authorizations.xml and users.xml file from my all nifi nodes.
Also There was confusion on about ‘OU’ in Node identities section.
What does it mean OU in node identities section? I don’t know yet.
Later I’ve mentioned ‘OU=nifi’ and also gave host names as ‘nifi1.abc.local’ , ‘nifi2.abc.local’, etc.
I have added AD/LDAP user in Initial Admin Identity(cn=architecture dev team,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com)
After setting above all, I was facing an error about setting nifi.security.identity.mapping.pattern.dn.
There was a challenge about the pattern definition.
There was 4 ‘ou’ I have defined in initial admin identities and login-identity-providers.xml.
So I’ve used below pattern and it worked well.
^cn=(.?),ou=(.?),ou=(.?),ou=(.?),ou=(.?),dc=(.?),dc=(.?)$
Note: I have removed Ranger completely.
Thanks,
Suraj
LikeLike
Hey, nice tutorial up there! I have a question if you may.
I followed the steps as they are and stuck at adding policies to new users I create new users logged in as Initial Admin through certificate. following are the policies in authorizations.xml for the initial admin.
I added users from the UI (Hamburger Icon). And then when I click on policies, I dont see newly created users in the list but only the Initial Admin.
Let us say I have to grant view the UI access to newly created “test” user. How do I do it?
Because when I login with test user the UI says Insufficient Permissions, Unable to view the user interface. Contact the system administrator.
Also followed the answer here
https://community.hortonworks.com/questions/91730/user-management-nifi.html
But, I cannot see the add user icon in the screenshot #2 on the link above. And dont even see the key icons next to newly added “admin” and “test” users.
Hope to hear from you soon.
LikeLike
Hi,
Do you have screenshots of what you’re seeing when you’re going into policies view? Can you share your authorizations.xml/users.xml files?
LikeLike
Hello, thanks for reply.
Here is the Policies View https://ibb.co/kvUwum
The authorizations file looks like https://ibb.co/bKC5n6
The users file https://ibb.co/mL89fR
And the authorizers file https://ibb.co/eSTNZm
LikeLike
With what user are you connected when you are in the policies view? I’m a bit surprised by what you’re seeing because the screenshot shows that admin, test and cornelius have read access to /flow but it’s not the case in the authorizations.xml file where we only see cornelius user.
LikeLike
I am logged in with Initial Admin, authenticated through a certificate.
LikeLike
Could you try removing the initial user identity in your user group provider definition? I don’t think that’s necessary since you already defined the user as initial admin identity. Also, delete users.xml and authorizations.xml files before restarting NiFi to be sure the files are recreated correctly.
LikeLike
Doing so ends up giving me an error
org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable to locate initial admin CN=cornelius, OU=MIC3 to seed policies
LikeLike
OK, my bad then. Let me try to reproduce on my side… In the meantime I’d suggest you subscribing to the users mailing list of NiFi and ask the question there. Others from the community might be able to help.
LikeLike
I have posted a question here too https://community.hortonworks.com/questions/155141/adding-policies-to-newly-added-users-by-initial-ad.html
If you find something please do add it there.
LikeLike
OK I think I understood what’s going on. I believe that if you want to define a user group provider that is configurable from the UI, you have to go through the Composite Configurable User Group provider definition. In your authorizers.xml file you’d have something like:
file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml
admin
composite-configurable-user-group-provider
org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider
file-user-group-provider
file-access-policy-provider
org.apache.nifi.authorization.FileAccessPolicyProvider
composite-configurable-user-group-provider
./conf/authorizations.xml
admin
managed-authorizer
org.apache.nifi.authorization.StandardManagedAuthorizer
file-access-policy-provider
LikeLike
[…] can find the usual parameters that you configured for the LDAP authentication part, but there is also a lot of new parameters to only synchronized specific parts of your remote […]
LikeLiked by 2 people
With respect to the ‘Identity Strategy’, there is room for improvement in Nifi…
When the strategy “USE_DN” is chosen, things should work fine until the point that the directory structure is changed and users get new DNs. Then the mappings would be broken.
Then the strategy “USE_USERNAME” is chosen, the issue with changing DNs is avoided, but two issues are introduced:
– The directory must be guaranteed to be free of duplicate usernames or one mapping will potentially refer to more than one user.
– In the case of a user deleted and another person being added with the same username, it is possible that the new user will unintentionally be given access.
It might be worthwhile to introduce a third strategy (let’s call it “USE_UUID”). The third strategy should define not only the “USE_UUID” setting but also provide a setting for an LDAP user object attribute which is unique, immutable, and never re-used. For instance, there is the Active Directory ‘objectGUID’ attribute or the OpenLDAP ‘entryUUID’ attribute.
Microsoft has some information about the objectGUID attribute here: https://docs.microsoft.com/en-us/windows/desktop/ad/using-objectguid-to-bind-to-an-object
LikeLike
Hi Jim, that’s a valid point and I invite you to create a JIRA for that. That’s certainly something that could be easily added into the framework. Thanks for the comment!
LikeLike
Thanks for the suggestion, Pierre. I’ve opened NIFI-5636. 🙂
LikeLike
your blog is very helpful!! i’m appreciated to you, and i have a question.
when i logging in Nifi this error message showed me up
“The supplied username and password are not valid.”
i checked this command line on terminal
” ldapsearch -D ‘cn=ldap_admin,dc=mapr,dc=com’ -w **** -p 389 -h myhost -b ‘dc=mapr,dc=com’ -s sub ‘uid=myuser’ ”
it’s perfectly working. that line printed to me myuser’s information
# myuser, people, mapr.com
dn: cn=myuser,ou=people,dc=mapr,dc=com
~~~~~~~~~~~
sn: myuser
uid: myuser
~~~~~~~~~~~~~
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
my identity strategy property is setting USE_USERNAME
but, using username : myuser and passwd : &&&& to login nifi is failed
my passwd is so simple, so i did not typing wrong.
what am i missing? i’m using centos7, nifi-1.8.0 and three node clustering.
before setting LDAP login provider, i used cert and so already have a CN=admin, OU=NIFI user.(also have node identity 1,2,3 because of clustering..)
if i have to change username on ldap? .. ldap have username “admin”(cn=admin,ou=people… same-thing on your article.) and cert have username ‘CN=admin, OU=NIFI’ ..
i’ll be thanks to you if you reply to this question!!
LikeLike
Hi. Could you share your ldap-provider configuration in a gist somewhere (after removing sensitive info)? I think there must be something wrong in here.
LikeLike
Hi, thank you for replying. this is my login-identity-provider.conf(https://gist.github.com/hr-pyo/61dbe8c717119a8ffe4718fe086b3885) . i’m using nifi1.8.0 , 3-node clustering and ldap server on node1,
my ldap.conf is
BASE ou=people,dc=mapr,dc=com
URI ldap://myhost:389
all node have “node1ip myhost ” line in /etx/hosts fiie.
#ps -ef | grep [s]lapd
ldap 766 1 0 12월21 ? 00:00:00 /usr/sbin/slapd -u ldap -h ldapi:/// ldap:///
# netstat -nlp -tu | grep slapd
tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 766/slapd
tcp6 0 0 :::389 :::* LISTEN 766/slapd
need something more? it’s my first time setting ldap server…
LikeLike
this is nifi-user.log
INFO [NiFi Web Server-210] o.a.n.w.a.c.IllegalArgumentExceptionMapper java.lang.IllegalArgumentException: The supplied username and password are not valid.. Returning Bad Request response.
LikeLike
Hi.. Thank you for the great tutorial, I followed the steps above . First, I created local certificate(CN=Adminstrator, OU=NIFI), then created local user “admin”. When I changed my configuration to use ldap, I got this error “Unknown user with identity ‘CN=Adminstrator, OU=NIFI’. Contact the system administrator.” When I connected to nifi.
LikeLike
You might want to double check your configuration and check users.xml and authorizations.xml files that are generated when NiFi is started for the first time. If in doubt and if you didn’t add custom authorizations/users, you can safely delete the file and restart NiFi. Files will be re-generated based on your configuration.
LikeLike
Hi, I want to do the same with Azure Active Directory instead of the local LDAP. My Nifi VM is also in Azure and want to integrate with Azure AD. Would you please guide me how to do that? Thank you.
LikeLike
Hi, I think you’d need to enable LDAP access to your AD following instructions like https://docs.microsoft.com/en-gb/azure/active-directory-domain-services/active-directory-ds-admin-guide-configure-secure-ldap
LikeLike
@pvillard31, Thanks for your prompt reply. Let me share my scenario, Actually We dont have any LDAP other than Azure Active. We have Tableau which authorized users through AZure AD. So I want to do the same for NiFi users through Azure AD. Can I do that without LDAP?
LikeLike
I’m running NIFI 1.8.0 and with you information from this page I have an almost working AD/LDAP integration. NIFI import users and groups (based on filter and searchbase) from our AD and also maps groups to users. So now we can assign policies to AD groups and manage users via AD.
But…
The background sync stops after 3-5 times. The org.apache.nifi.ldap doesn’t give any information even in debug or trace. I’v found that the package org.springframework.ldap write debug info about the background sync but I doesn’t have any errors messages.
Have anybody experience the same problems?
/Jens
LikeLike
Hi Jens, you might be running into https://issues.apache.org/jira/browse/NIFI-5935 if you can, I’d suggest upgrading NiFi to 1.9.0
LikeLike
@pvillard31, The article was really helpful and I was able to setup a local LDAP using Anonymous binding and NiFi instance.
But I am receiving this error when trying to login when using another LDAP server (I’ve changed the information required to match this LDAP server, and I can query AD using Anonymous binding in C#. I think it’s important to know that I am not LDAP/NiFi savvy and I don’t have admin access to it).
[LDAP: error code 1 – 000004DC: LdapErr: DSID-0C09075A, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1 ]; remaining name ‘ou=users,dc=xx,dc=yy,dc=zz’
Do you have any idea as to why?
LikeLike
I’m not familiar with this error. I’d try connection to the LDAP server from the command line using ldapsearch for example, just to be sure everything is OK.
LikeLike
Mr Villard , do you have an example that I can use to integrate the groups AD through login-identity-providers.xml ? when I try to add the AD groups it doesnt seem to work for me
LikeLike
Hi. Not sure to understand the question. Groups are more related to authorizations. Did you read https://pierrevillard.com/2017/12/22/authorizations-with-ldap-synchronization-in-apache-nifi-1-4/ ? If it does not help, feel free to ask your questions on the NiFi users mailing list.
LikeLike
Basically my question is : for LDAP-group based policies authorization on NIFI can I only accomplish it using Ranger ? Thanks
LikeLike
I don’t recall exactly from which HDF version it is possible, but yes it is. Best is to check on the community Hortonworks forum.
LikeLike
Hi @pvillard31, firstly I wanna thank you for the knowledge sharing with the community, this really helps me a lot
for some reason, i am able to access nifi Registry through SSL certs but not getting LOGIN page
did the below configs, not sure what did i miss
nifi.porperties:
nifi.security.needClientAuth=true
nifi.security.authorizers.configuration.file=./conf/authorizers.xml
identiy-providders.xml :
.
ldap-identity-provider
org.apache.nifi.registry.security.ldap.LdapIdentityProvider
SIMPLE
xxxxxx
xxxxx
FOLLOW
10 secs
10 secs
ldap://ldap.ad.xxx.xxx:389
dc=AD,dc=xxxx,dc=xxx
sAMAccountName={0}
USE_USERNAME
12 hours
LikeLike
What happens when accessing the NiFi Registry? Any error or message?
LikeLike
Hello All,
I could see few sections require more elaborate steps as per my knowledge:
I have done this AD integration with Apache Nifi – single server( ver. 1.7.0 ).
Here are the steps done and this could be useful for those who are doing this for first time as I did here:
On my Single Nifi Server we need to create certificates, truststores, keystores, etc and I have used openssl to do the same and I have my own CA.
https://community.cloudera./t5/Community-Articles/How-to-create-user-generated-keys-for-securing-NiFi/ta-p/245551
Check this for qa cluster:
https://community.cloudera.com/t5/Support-Questions/Remote-instance-of-NiFi-is-not-configured-to-allow-RAW/td-p/136978
Select a secure folder ( I used – opt/servercerts/ )on nifi-server and perform below steps:
a) Generate CA key
openssl genrsa -aes256 -out rootCA.key 4096
b) openssl req -x509 -new -key rootCA.key -days 3650 -out rootCA.pem
Inspect using: openssl rsa -check -in rootCA.key
c) openssl x509 -outform der -in rootCA.pem -out rootCA.der
d) keytool -import -keystore truststore.jks -file rootCA.der -alias rootCA
Check
keytool -v -list -keystore truststore.jks
Above steps will help to update below configurations and sample configuration is presented at the end of this section.
nifi.security.truststore=
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=
nifi.security.needClientAuth=true
Creating server keystore:
keytool -genkey -alias server2 -keyalg RSA -keystore nifi-server.jks -keysize 2048
keytool -certreq -alias server2 -keystore nifi-server.jks -file nifi-server.csr
openssl x509 -sha256 -req -in nifi-server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out nifi-server.crt -days 3650
keytool -import -keystore nifi-server.jks -file rootCA.pem
keytool -import -trustcacerts -alias server1 -file server1.crt -keystore nifi-server.jks
Above commands will help to provide values to below configuration and sample configuration is presented at the end of this section:
nifi.security.keystore =
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=
nifi.security.keyPasswd=
nifi.secuirty.needClientAuth=true
***********************
For the browser, convert nifi-server.jks to .p12 format using below command and add that .p12 certificate to browser and let browser chose the destination of cert based on the cert accordingly.
/jre/bin/keytool -importkeystore -srckeystore keystore.jks -srcalias cert1 -destkeystore keystore.p12 -deststoretype PKCS12
We have to add keystore.p12 to our browser ( In my case I used chrome )
************************
Now modify below configuration files:
a) authorizers.xml
b) nifi.properties
c) login-identity-providers.xml
In nifi.properties change properties as below:
nifi.remote.input.http.enabled=false ( we are not using http now )
#nifi.web.http.port=8082 ( I commented this instead of removing )
nifi.web.https.port=8082 ( https port )
nifi.security.keystore=/opt/servercerts/nifi-server.jks
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=
nifi.security.keyPasswd=
nifi.security.truststore=/opt/servercerts/truststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=
nifi.security.needClientAuth=yes
nifi.security.user.login.identity.provider=ldap-provider
Steps to generate keystore and truststore are mentioned above.
In authorizers.xml
I did add below at the end
file-provider
org.apache.nifi.authorization.FileAuthorizer
./conf/authorizations.xml
./conf/users.xml
CN=Hadoop,OU=Service Accounts,DC=xxx,DC=yyy,DC=COM
Please add above not at the very end.
Here, Hadoop is the service account or initial admin for Nifi and once after https is enabled, nifi will ask for username and password and in this case, I did use initial admin identity as Hadoop user present in our AD.
Replace CN accordingly as per your AD, I have masked DC details.
In login-identity-providers.xml, I did below
ldap-provider
org.apache.nifi.ldap.LdapProvider
SIMPLE
CN=Hadoop,OU=Service Accounts,DC=xxx,DC=yyy,DC=COM
Password
FOLLOW
10 secs
10 secs
ldap://ldap.xxx.yyy.com:389
OU=Service Accounts,DC=xxx,DC=yyy,DC=COM
sAMAccountName={0}
USE_USERNAME
12 hours
Restart Nifi.
Hope this helped, if not please comment and let me check whenever I get time.
Note: I might missed few steps, if so please update in comments section.
LikeLike
Hello,
Is it necessary some configuration at authorizers.xml file?
Thanks.
LikeLike
As soon as authentication is configured, configuration for authorisations is also required.
LikeLike
Yes, it is required at least in my case.
LikeLike