Building Directory Services with Net::LDAP

Joe Johnston, jjohn@cs.umb.edu

Why You Need Directory Services

Your little black book has become threadbare over the years from frenetic discothequeing, tardy nocturnal appeals to friends for bail money, and other shady Saturday night activities best left to the godless Carter years of double digit inflation and oil embargos. These days, you're on the go. You're mobile. You're wireless. You can't be tied down to one workstation, but you need your address book to follow you. After all, these are the days of robot maids and personal jet packs, right?

Perhaps you're the type of person who realizes it's not all about you. You might be an administrator for a workgroup whose members all need access to the same set of email addresses and aliases. Maybe your workgroup users have an eclectic set of email clients, like Eudora, Outlook, and Netscape. Do you want to maintain three separate address books for each client and then replicate the changes to each workstation? Only if you relish pain.

Centralized network address books, which can be used by client email programs like Outlook or Netscape Mail, are just one example of the kinds of programs that can be built with LDAP (Lightweight Directory Access Protocol). In this article, we'll look at what goes into building just such a program using Perl.

What Is LDAP?

LDAP is a protocol for directory services (database systems designed to allow fast searching against records all in a similiar format). LDAP is optimized to serve information that is frequently requested but rarely changed. Address books are the most common type of LDAP application.

Because modifications to the data are supposed to be infrequent, transactional database logic is not implemented by most LDAP servers. LDAP's emphasis is on speed, not robustness.

LDAP servers store data in a Directory Information Tree (DIT), which is a hierarchical grouping of related data. LDAP clients access this information for the user.

To make this more concrete, let's build a directory of national restaurants using an LDAP system. Users would be able to get a list of restaurants given a city or state using an LDAP client.

Each restaurant's information would be stored in the LDAP's DIT structure called an entry. Each entry has several fields, like "name", "telephone", "city", or "state". Here's an example of a very skeletal restaurant entry in LDAP Data Interchange Format (LDIF), which is commonly used to move text data into and out of LDAP servers:

   dn:          o=Canestaro, l=Boston, st=MA
   o:           Canestaro
   l:           Boston
   st:          MA
   description: Fine eatery in the heart of the Fens
   telephone:   999 555 1234
   objectclass: top
   objectclass: organization
Listing 1

In order to be found in the DIT, an entry must have a unique Distinguishing Name, or dn for short. In database speak, the dn is record's key. The Distinguishing Name field found in every entry will be made up of one or more Relative Distinguishing Names, or RDN. An entry's RDN is a set of that entry's fields which uniquely identify it from the rest of its siblings. In the restaurant example, the RDN is the organizational name, "o=Canestaro". This restaurant is in the city of Boston, in the state of Massachusetts and should be grouped with the rest of Bean Town's bistros. To represent the city and state groupings, we need to add cities and states to our DIT. Here are the two entries needed to describe Boston, MA:

dn: st=MA
st: MA
objectclass: top
objectclass: name

dn: l=Boston
l: Boston
st: MA
objectclass: top
objectclass: name
Listing 2

We nearly have all the components necessary to describe the dn of our example restaurant. The order in which attributes are listed in the dn are from the most entry specific to the least. Our example dn describes the name of the restaurant in its home city, which is located in some state.

The rest of our restaurant's entry description are a series of hash-like name-value pairs called attributes. Unlike the dn which contains data used only by the LDAP server, attributes store the information that users care about. Attribute names are often terse case insensitive one or two letter combinations. You can find full descriptions of these in RFC 2256, but here 'o' means 'organization name', 'l' stands for 'locality' or city, and 'st' is short for 'state'. The 'description' field provides human readable text about the entry.

The final attribute is 'objectclass', which is simple informs the LDAP server what fields are allowable and which aren't. Again, RFC 2256 is a good resource to investigate your objectclass options. Like Perl's OO object, LDAP objectclasses are heirarchical. Unlike Perl's objects whose parentage can be found in the @ISA array, LDAP entries must list all of its parent classes. All objeclasses are descended from 'top'.

Both OpenLDAP and Netscape's LDAP come with a command line utility called ldapsearch which allows us to examine our DIT. The basic arguments to ldapsearch are a search filter, which contains our search criteria, and an option list of fields to display. We will discuss how to build LDAP search filters in the section "A Searchable Web Interface To Manage Your Directory". To find all our Boston restaurant options, we could type:

% ldapsearch "(&(l=Boston)(st=MA)(o=*)" o telephone
Listing 3

This would give us output like this:

o=Canestaro, l=Boston, st=MA
o=Canestaro
telephone=999 555 1234

o=Boston BeerWorks, l=Boston, st=MA
o=Boston BeerWorks
telephone=999 555 1235
Listing 4

Obviously, this list has been shortened a bit (there is at least one more Boston restaurant).

This tells us what information our LDAP server will contain, so now let's examine the server itself.

Setting Up An OpenLDAP Server

I recommend the open source LDAP server from the OpenLDAP Project, at http://www.openldap.org. Follow the supplied directions for compiling ( ./configure and make ).

As mentioned above, OpenLDAP comes with a variety of command line tools, which all start with the word 'ldap', to manipulate our LDAP DIT. We have already seen ldapsearch. We could put our data into LDIF format and use ldapadd populate our DIT. Check your local manpages for usage options.

The standalone LDAP daemon ( slapd ) is what provides access to our DIT. The most important item required by slapd is a definition of the suffix field, which is used for the backend storage system used by LDAP. There must be at least one suffix field defined in the configuration file. The suffix is the top most RDN in which the rest of the DIT is contained. All our entries will need to have this RDN tacked on to their dn line. Here's what our suffix will be defined as:

  suffix "dc=daisypark, dc=net"
Listing 5

No authorization is required by our LDAP server to read the DIT. It is there to be a public resource, much like a web server. In order to make changes to the DIT, we need an administrative account. Not surprisingly, this account will be stored in our DIT as an entry. This account is normally called Manager and ought to have a decent password:

  rootdn "cn=Manager, dc=daisypark, dc=net"
  rootpw s3cr3t
Listing 6

The password is in clear text in the configuration file, and worse, it's passed to the server unencrypted. The specification for LDAP 3 details two authentication schemes. The first, called 'simple', transmits passwords in clear text. The other, Simple Authentication and Security Layer, is a protocol for plugging in our choice of authentication schemes. If your DIT will be updated over a public network, that's what you'll want to use.

One last detail is to create an entry for the which corresponds to the suffix that slapd requires. Recall that the suffix is tacked on to each and every entry as an RDN. All RDNs have a entry. Here is the one for our example DIT:

  dn:  dc=daisypark, dc=net
  dc:  daisypark
  dc:  net
  o:   Testing, Inc
  objectclass: top
  objectclass: organization
  objectclass: dcobject
Listing 7

Now that the configuration file is tailored to our system, let's start the server and populate it with our sample address data.

Loading Data Into The Directory

Graham Barr and Mark Wilcox have produced an excellent object-oriented interface to client LDAP operations called Net::LDAP, which should work with all standard LDAP servers. Let's build an LDAP client with this module that populates a DIT.

Our sample data is in tab-separated format, with the field names occupying the first line of the file. We'll assume that our field values won't have any embedded tabs, so parsing the file will be easy.

The interesting fields in this text file are FirstName, LastName, EmailAddress, HomePhone, Address, PostalCode, and StateOrProvince. We could use all of this information for each entry, but we will only be looking at names, email address, and phone numbers.

In the address loader (Listing 8), we'll focus on the code responsible for accessing the LDAP server. Those that have worked with DBI may find similiarities in the way Net::LDAP operates: connect to a server, do something, and then disconnect.

1   #!/usr/bin/perl --
2   # Script to transform tab delimited address data into LDAP.
3
4
5
6   use Net::LDAP;
7   use strict;
8
9   my $infile = $ARGV[0] || "addresses.txt";
10
11  my $conn = Net::LDAP->new("ldap.daisypark.net") or # Replace with your LDAP server
12    die "ERROR: Can't connect: $@";
13
14  # make a authenticated connection
15  $conn->bind( dn => 'cn=Manager, dc=daisypark, dc=net',
16               password => 'secret',
17             );
18
19  my $progress = 1;
20  $|++;
21  while ( my $rec = get_next_record($infile) ) {
22
23      next unless "$rec->{FirstName}$rec->{LastName}";
24
25      my $result = $conn->add(
26        dn => "cn=$rec->{FirstName} $rec->{LastName}, dc=daisypark, dc=net",
27
28        attr => [
29             cn              => "$rec->{FirstName} $rec->{LastName}",
30 
31             sn              => $rec->{LastName},
32             mail            => $rec->{EmailAddress},
33             telephoneNumber => $rec->{HomePhone},
34             street          => $rec->{Address},
35             postalCode      => $rec->{PostalCode},
36             st              => $rec->{StateOrProvince},
37             l               => $rec->{StateOrProvince},
38             c               => 'US',
39             objectclass     => [ 'top', 'person', 
40                                  'organizationalPerson', 'inetOrgPerson',
40                                ],
41               ],
42      );
43    
44
45      if ( $result->code ) {
46          warn "WARN: Failed to add entry: $rec->{FirstName}, $rec->{LastName}: ",
47          sprintf "%x", $result->code;
48      }
49
50      printf "seen: %d\r", $progress++;
51  }
52
53  print "\nClosing LDAP connection\n";
54  $conn->unbind;
55  print "done\n";
56
57  #------
58  # Subroutines
59  #------
60  # get_next_record() opens the tab-delimited file, returning successive
61  # hashref records, one per entry.
62  {
63    my ($seen, @headers); # persistent variables
64    sub get_next_record {
65        my $file = shift || return;
66
67        unless ( $seen ) {
68            open F, $file or die "ERROR: Can't open $file: $!";
69            @headers   = split "\t", scalar ;
70            $seen = 1;
71        }
72
73        my $line;
74        unless ( defined ($line = <>) ) {
75            close F;
76            return;
77        }
78
79        until ( $line ) {
80            chomp $line;
81            $line =~ s/\s*$//;
82        }
83
84        my $record = {};
85
86        @$record{ @headers } = (split "\t", $line);
87
88        return $record;
89    }
90  }

Listing 8

After connecting to our LDAP server as Manager, we read in our tab-delimited data one line at a time. Each line is a record that will be transformed into a hash. Each hash will become an entry in the DIT.

Line 11 instantiates a new Net::LDAP object that expects to find an LDAP server on a machine called daisypark.net. Recall that we can make changes, like adding entries, to the DIT only if we are the authorized user "Manager". We use the Net::LDAP bind method to "login" to the LDAP server as this account. We pass the password and the full I of the Manager account, cn=Manager, dc=daisypark, dc=net, to the bind method to prove that we are who we say we are.

As we fetch more records from our text file, we will add new entries to the DIT with the appropriately labeled add method. Line 25 does this with the parameters dn and attr. We put the attributes of the entry in an anonymous array of name-value pairs. To determine whether the add fails, we must examine the object returned by this method. This Net::LDAP::Message object, labeled in our script as $results, has a method called code. If this method returns a non-zero value, an error occurred during the add call.

Once we've read through the source text file, we do a little object cleanup on line 54 by closing down the connection to the LDAP object with the unbind method.

After running Listing 8 against our flat text address file, we ought to vet the data in the DIT for errors. There are a couple of ways to do this. LDAP servers typically come with tools to examine the DIT, like the previously mentioned ldapsearch. We can also use the Netscape 4.x Mail address book, itself an LDAP client, to look at our LDAP server. Either way, we can perform various searches until we are satisfied all went well with the data loading.

We would be done if we only needed to setup a company-wide rolodex to which existing user email clients (and their accompanying LDAP address books) could connect. This may be all a user ever wants to do with LDAP. However, we can do more with the Net::LDAP module. Let's build a searchable, editable web client interface to this DIT.

A Searchable Web Interface To Manage Your Directory

Our CGI application (Listing 9) is pretty run-of-the-mill, but you can tailor this code for your production needs. Listing 9 creates a form in which the user can enter a search term; it then displays all the matches found as an HTML table. Each row of the table can be edited and saved back into the DIT.

Perl has a wealth of modules to make a programmer's life easier, and Lines 5 through 9 demonstrate some familiar standbys: CGI, CGI::Carp, CGI::Pretty, and Net::LDAP. Efficiency wonks will note that the nicely formatted HTML output of CGI::Pretty it both slow and unneccessary. However, it's nice being able to look at human readable HTML during development. Also great for debugging is the fatalsToBrowser option of CGI::Carp.

As before, we need to make an authenticated connections to the LDAP server in lines 19 through 21 in order to make changes to the DIT.

The script has three functions. It renders to the sceen or "paints" a blank HTML form prompting for a search term if none is given. If a search term is given, the script looks through our DIT and return the results to the generic paint function. If a entry is edited, it makes the requested change and repaints the screen.

Lines 26 to 49 contain an odd for loop. This is simply a switch statement that enumerates the three functions of the program. The script looks for the CGI variable "action" to determine which function to execute. The default function is to paint a blank form.

1   #!/usr/bin/perl --
2   # jjohn 6/2000
3   # A CGI interface to a LDAP server. 
4   
5   use strict;
6   use CGI         qw/:all *table/;
7   use CGI::Carp   qw/fatalsToBrowser/;
8   use CGI::Pretty qw/:all/;
9   use Net::LDAP;
10
11  # first, set up the main objects
12  my $cgi     = CGI->new();
13
14  my $base_dn = 'dc=daisypark, dc=net';  
15  my $conn    = Net::LDAP->new("ldap.daisypark.net") or
16    die "ERROR: Can't connect: $@";
17
18  # Make a authenticated connection with the s3cr3t password
19  $conn->bind( dn       => "cn=Manager, $base_dn",
20               password => 's3cr3t',
21             );
22
23  # Have we been asked to do anything?
24  my $action = $cgi->param('action');
25
26  for ($action){
27      my $message = '';
28      /search/ && do {
29        $message = search(
30                          ldap    => $conn,
31                          base_dn => $base_dn,
32                          cgi     => $cgi,
33                         );
34      };
35
36      /modify/ && do {
37        $message = modify(
38                          ldap    => $conn,
39                          base_dn => $base_dn,
40                          cgi     => $cgi,
41                         );
42      };
43
44      paint( ldap    => $conn,
45             base_dn => $base_dn,
46             cgi     => $cgi,
47             message => $message,
48           );
49  }
50
51  $conn->unbind;
52  exit;
53
54  #-------
55  # Subroutines
56  #-------
57  sub paint{
58      my %params = @_;
59
60      print
61        header,
62        start_html( -bgcolor => "#FFFFFF",
63                    -title   => ($params{title} ||
64                                'View LDAP for Daisypark'),
65                  ),
66        h2( ($params{title} || 'View LDAP for Daisypark') ),
67        hr,
68        ($params{message} || 'Please perform a search');
69
70      print
71        hr,
72        start_form,
73        '<INPUT TYPE="HIDDEN" NAME="action" VALUE="search">',
74        'Search: ',
75        textfield( -name => 'search'),
76        submit,
77        end_form,
78        end_html;
79  }
80
81  # ldap_lookup() returns an LDAP entry object
82  # for the provided search term.
83  sub ldap_lookup {
84      my %params = @_;
85
86      my $criteria = $params{search};
87      my $filter;
88
89      # hack, I want to search for everything
90      # if I have an empty string
91      undef $criteria if $criteria eq '';
92
93      for (qw/c mail sn cn telephonenumber/) {
94          # todo: $params{search} needs to escape meta-chars!
95          if ( defined $criteria ) {
96              $filter .= "($_=*".$criteria."*) ";
97          } else {
98              $filter .= "($_=*) ";
99          }
100     }
101     $filter = "(| $filter)";
102
103     my $mesg = $params{ldap}->search(
104                                      base   => $params{base_dn},
105                                      filter => $filter,
106                                     );
107     if ( $mesg->code ) {
108         die "Oops ($filter): ", $mesg->error;
109     }
110
111     return $mesg;
112  }
113
114  # search() performs a lookup in the LDAP for a given string,
115  # returning a nice HTML table.
116  
117  sub search {
118      my %params = @_;
119
120      my $mesg = ldap_lookup( @_,
121                              search => ( $params{search} ||
122                                          $params{cgi}->param('search')
123                                        ),
124                            );
125
126      if ( $mesg->count == 0 ) {
127          return "No matches found for '$params{search}'";
128      }
129      my $results;
130      $results .= p(small('Matches: ' .
131                          b($mesg->count) .
132                          ' for term ' .
133                          b( $params{cgi}->param('search') )
134                   ));
135      $results .= start_table( -cellspacing => 0,
136                               -cellpadding => 0,
137                             );
138      $results .= Tr(
139                     th({-bgcolor=>'pink'},
140                        [qw/Name E-Mail Phone Change/])
141                    );
142      # add some pretty color every third row
143      my $row = 0;
144      for my $entry ( $mesg->all_entries ) {
145
146          my $cn = $entry->get('cn')->[0];
147
148          $results .= Tr(
149                         {-bgcolor => (!($row%3) ? "#CCCCCC" :"#FFFFFF") },
150                         start_form,
151                         '<INPUT TYPE="HIDDEN" NAME="action" VALUE="modify">',
152                         qq/<INPUT TYPE="HIDDEN" NAME="old_cn" VALUE="$cn">/,
153                         td(textfield( {   -name => 'cn',
154                                        -default => $entry->get('cn')
155                                        })),
156                         td(textfield( {-name    => 'mail',
157                                        -default => $entry->get('mail')
158                                   })),
159                         td(textfield( {-name    => 'telephonenumber',
160                                        -default => $entry->get('telephonenumber')
161                                   })),
162                         td( submit ),
163                         end_form,
164                        );
165
166          $row++;
167       }
168
169       return $results .= end_table;
170  }
171
172  sub modify {
173    my %params = @_;
174
175    my $old_cn = $params{cgi}->param('old_cn');
176    my $mesg = ldap_lookup( @_,
177                            search => $old_cn );
178
179    if ( $mesg->count == 0 ) {
180        return "Oops: Can't find $old_cn";
181    }
182
183    my $cgi = $params{cgi};
184    my $entry = $mesg->entry(0); # really need to iterate over results
185
186    # Delete if 'cn' is empty, else modify
187    my $report = '';
188    if ( $cgi->param('cn') =~ /^\s*$/ ) {
189        $entry->delete();
190        $report = "Deleted";
191    } else {
192        $entry->replace(
193                        cn              => $cgi->param('cn'),
194                        mail            => $cgi->param('mail'),
195                        telephoneNumber => $cgi->param('telephonenumber'),
196                       );
197        $report = "Updated";
198    }
199
200    $entry->update( $params{ldap} );
201    return $report . " " . $cgi->param('cn');
202  }

Listing 9

The first interesting function begins on line 83. ldap_lookup() queries the LDAP server for the given terms and returns the results as an LDAP message object.

RFC 2254 describes the many ways that LDAP can compare data. In Figure 9, we use only one of those ways: the partial case insensitive match. If the search term matches any part of the country, email, full name, or telephone fields, the search is considered successful. More refined search functions can easily be created.

Web client awaiting search term
Figure 1: Web client awaiting search term

LDAP requires a somewhat odd syntax for to describe search filters. Like anything the deals with a set of data, LDAP defines a group of boolean operators (e.g. "or" and "and"). These must precede the terms, which are just attribute name-value pairs, that they join. Fortunately, the operators themselves should look familiar:

  (| (c=US)(cn=joe*) )
   
  (& (c=US)(cn=joe*) )
Listing 10

In the first example, we are looking for entries in the DIT that have a country field of US or have canonical names that begin with joe. In the second example, we have the same terms "and"ed together, which selects only entries with both of those criteria.

The Net::LDAP search method returns a message object. We can check for error conditions by looking at the numeric code returned from the aptly named code() method. Any non-zero value indicates an error. The particular error message can be retrieved with a call to the error() method.

On line 120, the search terms are passed to ldap_lookup(). Assuming nothing fatal has happened, we then look at the message object's count() method, which returns the number of matched entries for our terms. Provided at least one entry matches, we can use a simple foreach loop to iterate over all the entry objects returned by the message object's all_entries() method.

This Entry object is our interface to an individual DIT entry. Because all fields in an entry can be mutli-valued, the get() method returns an anonymous list of values.

Spreadsheets are a useful metaphors for manipulating tabular data, such as this address book. Implementing this metaphor calls for some tricky HTML code. Lines 148 to 164 create one row representing one DIT entry. This row is an editable form that can update the entry if the user changes any values. Figure ??? shows the result of searching for an empty string. This is a special case in this application which returns the whole DIT. The number of entries in this address book was trimmed for this screenshot, but I specifically left my mother's name in the list to give her some reward for reading this article. She's not really into Perl and doesn't quite know what I do. (Mom, this is what I do.)

Web client displayingt the whole DIT
Figure 2: Web client displaying the whole DIT

Modifying an existing DIT node requires first locating the desired entry. This code locates entries by searching for the right cn field. This value was stored in a hidden field before the user made changes to this record. Here, I'll admit to a fudge. The search could return more than one entry. After all, cn is not exactly the same as the DN. There are no guarantees that the cn will be unique. A better value to have stored away, of course, is the DN of the desired entry. That's what it's there for.

If the user cleared out the cn field in the form, we will erase that entry. Otherwise, the script will call the Entry object's replace method to change the relevant fields. The Entry does not get updated on the LDAP server until the update method is called on line 200. Figure 3 shows the result of looking for bill. I'll add his email address and press the submit button. Figure 4 shows the dramatic results.

Entry before modification
Figure 3: Entry before modification

Entry after modification
Figure 4: Entry after modification

So there we have it. In about two hundred lines of code, we have a platform-independent address book. You can easily adapt the concept shown here to make a fabulous Perl/Tk version. One important note: this code does not ensure data integrity on updates. That is, if multiple users attempt to update the DIT at the same time, LDAP will make no attempt to lock its data. This sort of "Atomic Consistency Isolation and Durability" (ACID) support is well beyond LDAP's capabilities. If you find yourself needing it, use a real relational database management system.

Where LDAP Is Going

LDAP is becoming pervasive. It forms the backbone of both Microsoft's Active Directory system as well as Novell's Network Directory Services. Netscape has also been very active in developing their own LDAP server implementation, and even Sendmail, Inc. is supporting LDAP address systems. You'll likely see directory services mature and grow, eventually eclipsing such old network standards like NIS and possibly DNS.

__END__

Joe Johnston ( jjohn@oreilly.com ) has just discovered the joy and the pain of gnapster and has learned that just because he can download the entire Olivia Newton-John catalog doesn't mean he should. He created the Aliens, Aliens, Aliens web site ( http://aliensaliensaliens.com ).