MyVirtualDirectory Configuration & Deployment

A virtual directory is a service (typically provied via LDAP) that allows for the integration of multiple data repositories in a transparent manner to an application. There are several use cases for a virtual directory including (but not limmited to):

To learn how a virtual directory can help integrate your applications with your identity data view some virtual directory use cases.

A typical situation in many enterprise architectures is to maintain a seperate directory for internal and external users. Internal users being employees while the external directory may store contractors and supliers or even customers. In situations where all users need access to an applicaiton (such as a portal) the application may not support multiple directories. In this situation a virtual directory can be used to integrate the directories into a single namespace.

In the above example MyVD is used with two LDAP Inserts to create a single view of the two directories.

In certain situations its desirable to have a local directory for user information while an external directory is used for authentication. For instance a local directory for user data and ActiveDirectory for authentication. In order to achieve this there are two options:

Its not allways acceptable to utilize a password synchronization system for both policy and technical reasons. In this case a virtual directory can be used to delegate the authentication. In the below diagram the virtual directory sits as a proxy between the application and the local directory delegating all bind requests to the corporate ActiveDirectory utilizing Kerberos. This is not the only way to delegate authenticaiton but is one of the simplest.

While LDAP v3 is an IETF standard, not all applications abide by the standards. In addition to requiring certain attributes, some applications will not work if your directory does not conform to its requirements. Again as above either a virtual directory can be utilized or a new directory can be created based on a synchronization solution. In this example an application requires that users id attribute be called uid while your directory is a Microsoft ActiveDirectory (which stores the user’s id in the samAccountName attribute). In order to support this application MyVD can be configured to "rename" uid to samAccountName.

The above diagram shows the application performing a search with the filter "(uid=username)" that is transformed to "(samAccountName=username)". When the object is returned the attribute samAccountName is renamed to uid. This is a very basic but powerful example of how a virtual directory can perform data transformation.

Many organizations have a single enterprise directory that stores common user attributes and may be used for authentication. If an application requires additional attributes it’s generally very dificult to add these attributes to the enterprise directory. In this situation a virtual directory can be used to extend the enterprise directory by creating an edge directory that joins the enterprise directory to application specific attributes.

In the above example application attributes are stored in a relational database (as it may be easier to get an application specific database then application specific directory). The Joiner insert combines the database and the enterprise directory based the uid attribute.

In order to control the provisioning of users some organizations require users are updated via a common API or web service. A virtual directory can be used to call this web service when users are updated to integrate with the existing infrastructure. The below image depicts the integration of a web service through a custom insert to call the web sevice and a routing insert to redirect updates to the custom insert.

The above pieces of MyVD fit together to create a "flow" data. To illustrate this lets show how the following virtual directory would work:

In order to fulfill the above requirements a virtual directory will need to be built with:

We aren’t going to cover the specific configuration in this section, but we will show how these pieces fit together:

The above image shows all write operations (add, modify, delete rename) are directed towards the "Master" namespace while all read operations (search, modify) are sent to the DB namespace. This decision is made by the routing insert. Notice that the two namespaces have the same base (ou=people,dc=domain,dc=com). MyVD allows for namespaces to overlap. In this case there is no conflict because the routing plugin determines which namespace is utilized.

Once a namespace is determined, it’s chain is executed. The "Master" namespace has only a single insert which calls a custom web service to update a user’s profile. The "DB" namespace has two inserts on it’s chain. The first insert enables the use of Kerberos during the LDAP "bind" operation. Since this insert handles the bind operation, execution stops there. On searches and compares however the Kerberos insert "ignores" the request by passing it down through the chain to the database insert which is configured to work with the database storing user data.

The above example shows how a complex identity requirement could be implemented using MyVD. From here you can explore the various inserts that are currently provided or how to configure MyVD.

The MyVD configuration file is broken into three parts:

The listener portion defines what port the server will listen on and if there is a secure port. The global namespace defines what inserts will run for all requests and the local namespaces defines what inserts will run for individual namespaces.

The listener can be configured to listen on a secure port, a non secure port or both. To configure a non secure port simply supply a port number. In addition to the port number, you can specify a maximum number of entries or a maximum time limit (in milli-seconds). If not configured, the default is unlimited. The listener can be limited only on a particular host (default is 0.0.0.0). If authentication is required the simple authentication credential are needed to bind the listener (default false).

To open a secure port you need to know the port, have a keystore to provide a key for encryption and the password for that keystore.

Namespaces are configured by first specifying a list of inserts and then configuring each interceptor. To illustrate how to configure MyVD we will configure the virtual directory described in the MyVD overview. This virtual directory uses a relational database to store users while utilizing ActiveDirectory for authentication and a custom webservice for updating user information. Bellow is a diagram describing the virtual directory.

The global namespace is setup by first listing the inserts that are used in the global chain and then each insert is configured.

In the above diagram there a single "global" insert: the RoutingPlugin. This insert is used to instruct the router how to route certain requests and is described in the "Insert Reference" section. Based on the above diagram the global namespace configuration would be as follows.

MyVD is configured to have multiple "namespaces" which determine the flow of data through the system. Namespaces are separated by LDAP DNs. Like the global namespace, local namespaces contain chains of inserts. Unlike the global namespaces, local namespaces are separated by an LDAP DN and a weight to determine which namespace takes priority when there is a conflict.

To configure local namespaces:

In the above virtual directory there are two namespaces : Master & DB. The Master namespace has a custom built insert for updating identity data via a webservice. The DB namespace is configured with the kerberos insert and a database insert.

The first line defines the which namespaces will exist. Then each namespace and it’s chain are defined. There are two things to note about this configuration:

Below is the complete server’s configuration.

At this point you have been shown how to configure MyVD. From here you can look at the available inserts or how to build your own.

The access controls for MyVD are based on a draft RFC. The RFC can be read [[www.cnn.com] here]. The access control system utilizes a list of access control items. Each item is structured as:

The access log provides information about who is accessing MyVirtualDirectory. The access log provides the ability to roll when the log reaches a certain size or at a certain time.

This insert dumps the details of a request to the log. DumpTransaction is very useful when debugging a MyVD deployment. By placing this insert at various points in the server it’s possible to see how requests and responses are effected. Note that this insert can have an adverse effect on performance.

This insert should be placed in a namespace with an empty "nameSpace" property in order to construct a root dse object.

This insert is an example of how an extended operation could be implemented in MyVD. The insert reconstructs the operation in order to make sure that the request is routed properly.

The Master Replica Router is an insert that should be added to the global chain when you wish to seperate write operations from search operations, such as when utilizing MyVD in a master/replica environment.

This insert allows for a schema ldif file to be used as a schema entry

The Route by Attribute Value Router is an insert that should be added to the global chain when requests must be routed to a particular namespace based on an attribute value. An example would be when using MyVD to route requests to different directories based on an email suffix. Additionaly, a default route can be used when no other routes are available. This router is only executed on searches who’s filters are equality ("="). This filter is not applied to substring filters.

This insert should be installed on a namespace that should be allowed to fail when performing searches without causing an error to be returned to the client. When an error does occur, it is logged as a warning.

This insert ensures that the attributes requested by a search request are the only attributes returned to the user. This insert is usefull when additional attributes are added to an entry for meta data purposes but should not be returned to the user.

This insert allows for the addition of a static attribute value for a particular objectClass. It is useful for adding attributes to identify users based on their directory or namespace.

The attribute mapper insert is used to map the names of attributes. For instance it could map uid to samaccountname. This insert works on attribute names in both entries, dns and filters.

The attribute value mapper is used to map one entry value to another. For instance mapping "objectClass: inetOrgPerson" → "objectClass: user".

The composite attribute insert is designed to create a singe attribute from multiple attributes. Uses include creating a CN attribute from the givenName and sn as well as creating a uid from the same two attributes.

The DN attribute map the bases of DN attributes (such as uniqueMember or manager)

This insert allows for the RDN of an entry to be changed. For instance if the current RDN is the "CN" attribute this insert can change it to the "uid" attribute.

This insert will map an attribute storing a DN to an attribute value on the referenced object. For instance this insert could map the uniqueMember DN to the uid of the user object pointed to be the DN. This would be useful when mapping from a groupOfUniqueNames to a posixGroups object.

This insert will format directory value stored as the number of milliseconds since epoch.

The Attribute2DN insert is useful for translating an attribute value to a full DN by doing a search. This is useful when using MongoDB for storing groups where a user identifier is stored in a document instead of a full DN. This insert will then translate that attribute into a full DN by searching for the user named in the attribute.

This insert creates a virtual attribute for storing the DNs of objects that reference this object, such as the common memberof attribute. This attribute is read-only and can be searched on in a filter.

This insert will delete an attribute from an entry.

The joiner is used to combine two LDAP namespaces (note that this means two DN namespace, not two configuration namespaces). The joiner combines entries from each namespace.

The simple join modify insert is configured AFTER a joiner on a chain in order to support the modification of joined attributes

The join add flat namespace insert is designed to allow the addition of entries to a joiner namespace. NOTE: this insert MUST be configured as the last insert on a chain.

The LDAP insert is one of the core inserts. This insert allows MyVD to communicate with any LDAPv3 compliant directory. In addtion to LDAP directories, this insert supports DSMLv2 and SPML. In order to take advantage of the SPML feautures you must download the OpenSPML toolkit (http://www.openspml.org/) and it’s dependencies.

The kerberos authenticator can be used to authenticate users via the Kerberos protocol, such as against Active Directory. Note that all configuration is based on the Java JNDI kerberos authentication file. (need to add a link)

The kerberos authenticator can be used to authenticate users via the NTLM protocol, such as against Active Directory or an NT4 Domain Controller.

The Dynamic Groups Insert allows for a Dyanmic Group (groupOfUrls) to act like a static group in the following ways:

This all allows for applications that are not capable of using dynamic groups while still providing the maintenance benefits of static groups.

The Embedded Groups insert allows for static groups to contain members that are other groups to be tested with a user’s DN. For instance:

The above example would typically require two searches in order to test cn=Test User,ou=people,dc=domain,dc=com:

The Embedded Groups does this work for you, tracking which members are groups to determine what groups need to be searched. The insert can track which members are groups either by polling the remote directory periodically or (if the directory supports it) by using persistent search to constantly track the addition or removal of groups.

The Static Bind DN Map insert can perform a static mapping from an external DN to a DN for a remote directory. Typically this insert would be used to map from DNs that are specific for an insert’s base to a non-specific base. Such as when using Sun/Fedora Directory the root user is typically cn=Directory Manager which has no base, so this insert could map uid=admin,dc=domain,dc=com→cn=Directory Manager.

The JDBC insert is used to expose a relational database through MyVD. This insert only supports a single objectClass and rdn for each configuration of the insert.

This insert contains the logic to update a single database table. It is configured BEHIND a database insert in order to utilize that insert’s connection pool and mapping.

This insert is used when groups stored in a relational database are stored with a name and not a full DN. This plugin will create the DN to make it LDAP compliant.

Often with relational databases a full, or "common", name is not stored as a single field but is rather stored as a first, or given, name and a last, or sur, name. Since LDAP typically expects there to be a composite attribute that has both pieces in a single attribute. This insert can also be used to create user ids based on a person’s name.

By default the JdbcInsert does not support authentication. This insert, configured BEFORE the JdbcInsert allows a a simple authentication using an un-salted one-way hash.

By default the JdbcInsert does not support authentication. This insert, configured BEFORE the JdbcInsert supports LDAP binds against a password stored using the PBKDF2 format created by net.sourceforge.myvd.util.PBKDF2.generateHash.

The SPML identity insert is designed to create an LDAP compliant DN for use with the LDAP insert in SPML mode. SPML does not require a full DN as a name (it can for instance use a userid or an email address), but LDAP does. This insert ensures that the DN for the user object has the correct RDN for SPML while being LDAP compliant.

This insert removes components from an LDAP DN and can optionally store them in the user entry.

Nearly every object in Active Directory contains both an Object GUID and an Object SID. The GUID is a unique identifier in Active Directory, with the SID being a security identifier. These are both binary attributes that are easiest to represent as strings. This insert takes the binary version of the these attributes and re0works them to present the string based representation. It also takes the string based representation in filters and translates it to a binary version.

Every user has a primary group. This group is noted in the Active Directory attribute primaryGroupID, which references the last component of the of the primary group’s objectsid attribute. Users are not listed as members of their primary group. This insert, when combined with the "ObjectGUID & ObjectSID to String" insert beneath it and the DynamicGroups insert above it will list all members of a group, including those who note the group as its primary group.

This insert helps to integrate ActiveDirectory with posix systems. It is configured on the same chain as a joiner and provides the following functions:

The active directory insert takes a Microsoft Active Directory and makes it appear to be a standard inetOrgPerson type directory. This insert utilizes the LDAP insert in order to communicate with Active Directory and includes additional mapping functions:

While this insert isn’t directly associated with Active Directory, when combined with the Active Directory Joiner and the Active Directory insert this insert provides additional posix attributes.

The Active Directory / Posix Joiner is an insert that combines the general joiner with the mapping inserts and Active directory inserts in order to create a view for posix integration. This joiner is meant to join a namespace configured with the AD Insert and another namespace configured for users and groups using the PosixDB inserts.

This insert allows a client that tries to search on an ObjectGUID that has been cast to text improperly.

Active Directory “user” objects don’t all have user principal name objects which can interfere with directory based systems that expect them. This insert will create a userPrincipalName object based on a directory attribute and suffix.

The objectGUID attribute is a binary attribute that is often corrupted by translation to text. This insert will translate a binary attribute to text properly.

This insert will check the userAccountControl in AD for the value of "2" and set an attribute to "true" if the user is enabled and "false" if the user is disabled.

There are many reasons why you would create a custom insert. These range from integrating a resource that is not currently supported by MyVD (such as a web service) to needing to implement more complex routing rules then existing Inserts provide.

MyVD requires the following for developing a custom insert:

As for how you develop and integrate your custom insert, thats up to you.

Once your development environment is setup, creating an insert only requires that you implement the net.sourceforge.myvd.inserts.Insert interface. A bare bones insert might look like this:

Each method has essentially the same pattern:

\1. Pre-operation work (ie adding attributes or routing)

\2. Continue down the operations chain

\3. Post-operation work

If this insert is going to handle the request then don’t call the methods "nextXXX" method. The "configure" method is called when the insert is initiated. The "props" object contains all properties configured for this insert.

Sometimes it’s desirable to perform a search or update an object in the directory inside of an insert. Inserts may create new chains of execution via the "createXXX" methods on the chain object.

This code will execute every insert below the current one. So for instance if this insert is the 3rd in a chain of five inserts the above code would execute inserts four and five.

One feature that only exists in MyVD is to utilize the JDBC-LDAP bridge to use SQL inside of MyVD instead of coding what is needed uing the above method. This is useful for Java developers who are more familiar with SQL then LDAP. A connection object can be retrieved via the chain object.

Once the insert is complete, jar it and copy it to the "lib" directory of your MyVD installation. Then you can configure your insert like any other insert.

MyVD require JRE 1.8 or higher. All other pre-requisites come with MYVD.

Deploying MyVD is fairly straight forward. The download bundle has several directories:

The bin directory contains unix and windows startup scripts. To start MyVD on windows, double click (or create a shortcut to) myvd.vbs. On unix run the myvd.sh script with either the parameters "start" or "stop".

If you wish to create a custom start script the MyVD startup command is

The myvd.conf file in the conf directory of the MyVD bundle has a very basic default configuration. This configuration is setup to have the DumpTransaction insert on the global chain and a sample RootDSE configuration.

MyVD utilizes the Log4j2 (http://logging.apache.org/log4j/2.x/) logging system. By default MyVD will log to the "logs" directory in the MyVD bundle at an "informational" level. The full default Log4J configuration is

To customize logging log4j properties may be set in a file called log4j2.xml in the same directory as the myvd.conf file used to configure your instance. If you only wish to make a change to any of the above properties you only need to set that property. For instance to enable "debug" mode the log4j2.xml file should look like:

MyVD comes with start scripts for both *nix and Windows. In order to use these scripts two environment variables need to be defined:

The LDAP Insert is the most important insert for integrating directories as it acts as the primary interface between a remote directory and MyVD. In order to setup a simple proxy all one must do is create an LDAP insert. Below is the configuration information for the LDAP insert:

The LDAP insert is one of the core inserts. This insert allows MyVD to communicate with any LDAPv3 compliant directory. In addtion to LDAP directories, this isnert supports DSMLv2 and SPML. In order to take advantage of the SPML feautures you must download the OpenSPML toolkit (http://www.openspml.org/) and it’s dependencies.

The above configuration parameters are for the most part very straight forward. The host and port configs are the host and port the directory is running on. The remoteBase config is to determine where in the remote directory MyVD should connect to. THe minimumConnections and maximumConnections are for connection pooling. The proxyDN and proxyPass configuration options may be supplied if you do not want to proxy the user’s credentials. When these are set they are used for all operations except for the bind operation. The type parameter is used to determine the protocol type. The LDAP Insert is based on Novel’s JLDAP implementation which supports LDAPv3, DSMLv2 (LDAP Web Service) and SPML (provisioning Web Service). Finaly, the spmlImpl property is used to determine how an SPML insert should interact with the remote server. By simply seting up an LDAP insert using the above parameters you can easily setup a remote proxy.

The below configuration is from the unit tests and shows a very simple proxy:

The above configuration is very straight forward. A single insert is created for an ldap server running on the local server with administrative credentials. While the above example is simple, it is only the tip of the iceberg.

First lets setup a simple database insert. The database insert has the following configuration

We are going to integrate a simple database table that will contain the following columns

You will notice that there are three fields missing that are typically a included in any directory object: uid, cn and userPassword. Each of these fields have been left out because they generally do not exist in databases. Databases are rarely used for authentication so there is rarely a uid and password. Because databases tend to normalize data, they don’t store a full name in a single field. If this is a requirement, we will cover how to handle these cases later on.

The table we are going to work with is called "employees" with the following data:

The first step is to download MySQL and the MySQL jdbc drivers. Thisguide doesn’t cover setting up MySQL. Once you downloaded MySQL and have setup the database, copy mysql-connector-java-VERSION-bin.jar to the lib directory of your myvd installation. If you are using the MyVD startup script MyVD will automatically pickup the drivers. Once deployed we can create our configuration.

Once downloaded, the myvd.conf file needs to be configured to utilize the employees table. Below is the configuration we are going to use:

This configuration has four major parts:

The first three configuration points are a part of the standard sample myvd.conf file included in the download. The final configuration point is the heart of the matter. The driver, url, user and password configuration options general options. The rdn option is the LDAP name of the database field, in this case the empid LDAP attribute is mapped to the id database field. The mapping option is the ldap=db mapping. The left hand side of the equals is the LDAP name and the right side of the equals is the database field name. The objectClass configuration is what object class entries from this insert will have. This can be any object class. Finally the sql option is an SQL statement that provides all of the database fields. This can be a simple SELECT as in the above case or can be something more complex as long as every column listed in the mapping configuration is included.

Thats it. Thats all thats involved. Below are the more advanced topics involved in exposing a database via MyVD.

Unlike LDAP directories, relational databases do not allways store information in a final "view" state. The simplest such example of this is the common name, or "cn" attribute. Another such example is the "uid" attribute. In the below example, we are going to add to our simple DB insert to include both a uid an cn attribute.

When mapping attributes its important to understand how the mapping process works. There are two directions to every LDAP request. The first is the inbound request. On the inbound request the attributes requested and the filter used must be updated. For instance if you are requesting the "uid" attribute, which in our case is a compound attribute of the givenName and sn attributes, then the request must instead include a request for the givenName and sn attribute. If the filter for the request is "(uid=sperson)" then the filter must be transformed to "(&(givenName=S*)(sn=Person))". The outbound result then needs to construct the attribute from the components. Here’s an example. The below table shows the initial request and the final request:

MyVD includes an insert to perform all of this work for you called the composite attribute insert. This insert takes the attributes that are to be assembled and creates the attribute on outbound entries and adjusts all requests accordingly. Fist lets look at how teh composite attribute insert is configured:

The composite attribute insert is designed to create a singe attribute from multiple attributes. Uses include creating a CN attribute from the givenName and sn as well as creating a uid from the same two attributes.

The "attribute" parameter is pretty straight forward. The components configuration is a bit more comlex. The goal of this configuration option is to determine how an attribute is created. The value for this option is "attribute:length,attribute:length,…". The length can be either a number, "" to indicate the entire attribute, "fs" to indicate that the attribute goes until the first space of the final attribute and finally "ls" for the last space of the final attribute. For instance a uid would be constructed by the first letter of the first name and the last name, or "givenName:1,sn:". A cn would be the entire first name, a space and then the last name or "givenName:fs,sn*". Some organizations include the middle initial in the first name, which means you would want the last space or "givenName:ls,sn:*". The next configuration, "properCase", is used to automaticlly set the case from all lowercase to a proper case. This is useful when you want your uid’s to be lowercase. Finally the objectClass configuration is used to determine when the composite attribute should be built.

Now that we’ve covered how the composite insert works and how its configured, lets look at the configuration:

The above configuration is very similar to the original DB configuration. We have added three new inserts. The first insert ensures that the attributes returned on a search are the ones requested in the search. The second insert creates the UID attribute while the third insert creates a CN attributes.

Some applications require a particuler RDN attribute to work properly. This is usually true of applications that require the user id to be a part of the RDN. In our above example the "empid" attribute is used as the RDN. For our database this makes sence because the id is the most unique identifier for our users. However, we are integrating with an application requires the uid we constructed in the last tutorial to be our user’s RDN. In order to do this we must translate every inbound request and outbound response. There is an insert that does this work for us however called the SetRDN insert. Here is the SetRDN insert’s configuration:

This insert allows for the RDN of an entry to be changed. For instance if the current RDN is the "CN" attribute this insert can change it to the "uid" attribute.

The configuration is fairly straight forward. The internalRDN is the rdn stored in the directory while the externalRDN is what is presented to the user. The below configuration shows how the SetRDN insert can be added to our example:

Thats it. We have now exposed our directory with a new RDN attribute.

The JDBC insert is a read only insert. It is based on an SQL statement which can have almost anything in it, preventing any way of knowing how an update is processed. There are three main reasons why the JDBC insert does not manage updates to the database:

In order to update user entries MyVD allows you to create an insert that sits behind the JDBC insert that can be passed both the database-ldap mapping and a connection from the JDBC insert allowing the insert to to update the database. This insert can access three request variables:

This setup allows for MyVD to allow for database updates while not forcing you to manage database connections or mappings, re-using the configuration from the JDBC insert.

MyVD has a simple JBDC table update insert that can be used for updating a single table. This insert makes it easy to update the database you have integrated without writing custom code. Here is the configuration for the DBTableUpdate insert:

This insert contains the logic to update a single database table. It is configured BEHIND a database insert in order to utilize that insert’s connection pool and mapping.

This insert utilizes the DB insert as described above by adding the DBTableUpdate insert AFTER the DB insert. The tableName is the name of the table being updated while the dbInsertName is the name of the insert on this chain whose database pool and mapping configuration will be used.

Configuring this insert is very easy. The below configuration adds this insert to our above example of creating a uid and setting the rdn:

Now our example is updateable!

The JDBC Insert doesn’t provide any way to authenticate against the database. this is because there are no standards as far as databases are concerned on how to store passwords and authenticate users. Just as with performing writes to the database, MyVD can pass a custom insert the database connection and mapping information allowing the custom insert to perform the authentication in whatever manner is required by the database.

Databases and directories store data in very different ways. Databases store data in tables that "relate" to each other where as a directories store data as objects. This means that databases don’t store mutliple value for a field in a given row where a directory can. In order to handle this the jdbc insert utilizes a subquery to ensure that all attribute values are returned MyVD uses sub-selects to perform queries. The basic of the SQL is:

Depending on how your tables are indexed this can be very slow. If you aren’t able to index the proper fields or there are only single value attributes comming from the database you can use the "useSimple" flag on the jdbc insert to utilize a simpler SQL statement. This will improve performance but may sacrifice data consistency based on the type of filters being used.

As stated above the joiner is built to only directly support searches. This is because modifications may follow seperate rules and adds & deletes require knowing and understanding namespaces involved in a join. There are several two ways to perform writes to a joined namespace:

We will configure our virtual directory to be able to update both the enterprise and application specific directory. In order to support adds, modifies, deletes and renames we will configure several additional inserts:

The above inserts all have a few things in common. They are all configured AFTER their main insert and they relly on their main insert’s configuration. That is to say that the JoinAddFlatNS and SimpleJoinModify inserts are configured after the joiner insert and the DBTableUpdate insert is configured after the application database insert. In addition, they are configured to "know" about their main insert to eliminate double configurations. The below configuration will allow for the creation and modification of entries of the joiner we previously configured:

Now you can create users and modify them seamlessly through your virtual directory.