Joe Johnston, jjohn@cs.umb.edu
Your little black book has become threadbare over the years from frenetic discothequeing, tardy nocturnal appeals to friends for bail money, and other shady Saturday night activities best left to the godless Carter years of double digit inflation and oil embargos. These days, you're on the go. You're mobile. You're wireless. You can't be tied down to one workstation, but you need your address book to follow you. After all, these are the days of robot maids and personal jet packs, right?
Perhaps you're the type of person who realizes it's not all about you. You might be an administrator for a workgroup whose members all need access to the same set of email addresses and aliases. Maybe your workgroup users have an eclectic set of email clients, like Eudora, Outlook, and Netscape. Do you want to maintain three separate address books for each client and then replicate the changes to each workstation? Only if you relish pain.
Centralized network address books, which can be used by client email programs like Outlook or Netscape Mail, are just one example of the kinds of programs that can be built with LDAP (Lightweight Directory Access Protocol). In this article, we'll look at what goes into building just such a program using Perl.
LDAP is a protocol for directory services (database systems designed to allow fast searching against records all in a similiar format). LDAP is optimized to serve information that is frequently requested but rarely changed. Address books are the most common type of LDAP application.
Because modifications to the data are supposed to be infrequent, transactional database logic is not implemented by most LDAP servers. LDAP's emphasis is on speed, not robustness.
LDAP servers store data in a Directory Information Tree (DIT), which is a hierarchical grouping of related data. LDAP clients access this information for the user.
To make this more concrete, let's build a directory of national restaurants using an LDAP system. Users would be able to get a list of restaurants given a city or state using an LDAP client.
Each restaurant's information would be stored in the LDAP's DIT structure called an entry. Each entry has several fields, like "name", "telephone", "city", or "state". Here's an example of a very skeletal restaurant entry in LDAP Data Interchange Format (LDIF), which is commonly used to move text data into and out of LDAP servers:
dn: o=Canestaro, l=Boston, st=MA o: Canestaro l: Boston st: MA description: Fine eatery in the heart of the Fens telephone: 999 555 1234 objectclass: top objectclass: organization | |
| Listing 1 |
|---|
In order to be found in the DIT, an entry must have a unique
Distinguishing Name, or dn for short. In database speak,
the dn is record's key. The Distinguishing Name field
found in every entry will be made
up of one or more Relative Distinguishing Names, or RDN.
An entry's RDN is a set of that entry's fields which
uniquely identify it from the rest of its siblings. In the
restaurant example, the RDN is the organizational name,
"o=Canestaro". This restaurant is in the city of Boston,
in the state of Massachusetts and should be grouped with the
rest of Bean Town's bistros. To represent the city and state
groupings, we need to add cities and states to our DIT.
Here are the two entries needed to describe Boston, MA:
dn: st=MA st: MA objectclass: top objectclass: name dn: l=Boston l: Boston st: MA objectclass: top objectclass: name | |
| Listing 2 |
|---|
We nearly have all the components necessary to describe the
dn of our example restaurant. The order in which attributes
are listed in the dn are from the most entry specific to
the least. Our example dn describes the name of the
restaurant in its home city, which is located in some state.
The rest of our restaurant's entry description are a series
of hash-like name-value pairs called attributes. Unlike the
dn which contains data used only by the LDAP server, attributes
store the information that users care about. Attribute names
are often terse case insensitive one or two letter combinations.
You can find full descriptions of these in RFC 2256, but here
'o' means 'organization name', 'l' stands for 'locality' or city,
and 'st' is short for 'state'. The 'description' field provides
human readable text about the entry.
The final attribute is 'objectclass', which is simple informs
the LDAP server what fields are allowable and which aren't.
Again, RFC 2256 is a good resource to investigate your objectclass
options. Like Perl's OO object, LDAP objectclasses are heirarchical.
Unlike Perl's objects whose parentage can be found in the @ISA
array, LDAP entries must list all of its parent classes. All
objeclasses are descended from 'top'.
Both OpenLDAP and Netscape's LDAP come with a command line
utility called ldapsearch which allows us to examine our
DIT. The basic arguments to ldapsearch are a search filter,
which contains our search criteria, and an option list of fields
to display. We will discuss how to build LDAP search filters
in the section "A Searchable Web Interface To Manage Your
Directory". To find all our Boston restaurant options,
we could type:
% ldapsearch "(&(l=Boston)(st=MA)(o=*)" o telephone | |
| Listing 3 |
|---|
This would give us output like this:
o=Canestaro, l=Boston, st=MA o=Canestaro telephone=999 555 1234 o=Boston BeerWorks, l=Boston, st=MA o=Boston BeerWorks telephone=999 555 1235 | |
| Listing 4 |
|---|
Obviously, this list has been shortened a bit (there is at least one more Boston restaurant).
This tells us what information our LDAP server will contain, so now let's examine the server itself.
I recommend the open source LDAP server from the OpenLDAP Project, at
http://www.openldap.org. Follow the supplied directions for compiling
( ./configure and make ).
As mentioned above, OpenLDAP comes with a variety of command line tools,
which all start with the word 'ldap', to manipulate our LDAP DIT.
We have already seen ldapsearch. We could put our data into LDIF
format and use ldapadd populate our DIT. Check your local manpages
for usage options.
The standalone LDAP daemon ( slapd ) is what provides access to our DIT.
The most important item required by slapd is a definition of the
suffix field, which is used for the backend storage system used
by LDAP. There must be at least one suffix field defined in the
configuration file. The suffix is the top most RDN in which the
rest of the DIT is contained. All our entries will need to have
this RDN tacked on to their dn line. Here's what our
suffix will be defined as:
suffix "dc=daisypark, dc=net" | |
| Listing 5 |
|---|
No authorization is required by our LDAP server to read the
DIT. It is there to be a public resource, much like a web server.
In order to make changes to the DIT, we need an administrative account.
Not surprisingly, this account will be stored in our DIT as an
entry. This account is normally called Manager and ought to
have a decent password:
rootdn "cn=Manager, dc=daisypark, dc=net" rootpw s3cr3t | |
| Listing 6 |
|---|
The password is in clear text in the configuration file, and worse, it's passed to the server unencrypted. The specification for LDAP 3 details two authentication schemes. The first, called 'simple', transmits passwords in clear text. The other, Simple Authentication and Security Layer, is a protocol for plugging in our choice of authentication schemes. If your DIT will be updated over a public network, that's what you'll want to use.
One last detail is to create an entry for the which corresponds to
the suffix that slapd requires. Recall that the suffix is tacked
on to each and every entry as an RDN. All RDNs have a entry.
Here is the one for our example DIT:
dn: dc=daisypark, dc=net dc: daisypark dc: net o: Testing, Inc objectclass: top objectclass: organization objectclass: dcobject | |
| Listing 7 |
|---|
Now that the configuration file is tailored to our system, let's start the server and populate it with our sample address data.
Graham Barr and Mark Wilcox have produced an excellent object-oriented
interface to client LDAP operations called Net::LDAP, which should work
with all standard LDAP servers. Let's build an LDAP client with this
module that populates a DIT.
Our sample data is in tab-separated format, with the field names occupying the first line of the file. We'll assume that our field values won't have any embedded tabs, so parsing the file will be easy.
The interesting fields in this text file are FirstName,
LastName, EmailAddress, HomePhone, Address, PostalCode,
and StateOrProvince. We could use all of this information for each
entry, but we will only be looking at names, email address, and phone
numbers.
In the address loader (Listing 8), we'll focus on the code responsible
for accessing the LDAP server. Those that have worked with DBI may find
similiarities in the way Net::LDAP operates: connect to a server, do
something, and then disconnect.
1 #!/usr/bin/perl --
2 # Script to transform tab delimited address data into LDAP.
3
4
5
6 use Net::LDAP;
7 use strict;
8
9 my $infile = $ARGV[0] || "addresses.txt";
10
11 my $conn = Net::LDAP->new("ldap.daisypark.net") or # Replace with your LDAP server
12 die "ERROR: Can't connect: $@";
13
14 # make a authenticated connection
15 $conn->bind( dn => 'cn=Manager, dc=daisypark, dc=net',
16 password => 'secret',
17 );
18
19 my $progress = 1;
20 $|++;
21 while ( my $rec = get_next_record($infile) ) {
22
23 next unless "$rec->{FirstName}$rec->{LastName}";
24
25 my $result = $conn->add(
26 dn => "cn=$rec->{FirstName} $rec->{LastName}, dc=daisypark, dc=net",
27
28 attr => [
29 cn => "$rec->{FirstName} $rec->{LastName}",
30
31 sn => $rec->{LastName},
32 mail => $rec->{EmailAddress},
33 telephoneNumber => $rec->{HomePhone},
34 street => $rec->{Address},
35 postalCode => $rec->{PostalCode},
36 st => $rec->{StateOrProvince},
37 l => $rec->{StateOrProvince},
38 c => 'US',
39 objectclass => [ 'top', 'person',
40 'organizationalPerson', 'inetOrgPerson',
40 ],
41 ],
42 );
43
44
45 if ( $result->code ) {
46 warn "WARN: Failed to add entry: $rec->{FirstName}, $rec->{LastName}: ",
47 sprintf "%x", $result->code;
48 }
49
50 printf "seen: %d\r", $progress++;
51 }
52
53 print "\nClosing LDAP connection\n";
54 $conn->unbind;
55 print "done\n";
56
57 #------
58 # Subroutines
59 #------
60 # get_next_record() opens the tab-delimited file, returning successive
61 # hashref records, one per entry.
62 {
63 my ($seen, @headers); # persistent variables
64 sub get_next_record {
65 my $file = shift || return;
66
67 unless ( $seen ) {
68 open F, $file or die "ERROR: Can't open $file: $!";
69 @headers = split "\t", scalar ;
70 $seen = 1;
71 }
72
73 my $line;
74 unless ( defined ($line = <>) ) {
75 close F;
76 return;
77 }
78
79 until ( $line ) {
80 chomp $line;
81 $line =~ s/\s*$//;
82 }
83
84 my $record = {};
85
86 @$record{ @headers } = (split "\t", $line);
87
88 return $record;
89 }
90 }
| |
| Listing 8 |
|---|
After connecting to our LDAP server as Manager, we read in our
tab-delimited data one line at a time. Each line is a record that will
be transformed into a hash. Each hash will become an entry in the DIT.
Line 11 instantiates a new As we fetch more records from our text file, we will add new entries
to the DIT with the appropriately labeled Once we've read through the source text file, we do a little object
cleanup on line 54 by closing down the connection to the LDAP object
with the After running Listing 8 against our flat text address file, we
ought to vet the data in the DIT for errors. There are a couple
of ways to do this. LDAP servers typically come with tools to
examine the DIT, like the previously mentioned We would be done if we only needed to setup a company-wide rolodex
to which existing user email clients (and their accompanying LDAP
address books) could connect. This may be all a user ever wants
to do with LDAP. However, we can do more with the Our CGI application (Listing 9) is pretty run-of-the-mill, but you can
tailor this code for your production needs. Listing 9 creates a form
in which the user can enter a search term; it then displays all the
matches found as an HTML table. Each row of the table can be
edited and saved back into the DIT.
Perl has a wealth of modules to make a programmer's life easier, and
Lines 5 through 9 demonstrate some familiar standbys: As before, we need to make an authenticated connections to the LDAP
server in lines 19 through 21 in order to make changes to the DIT.
The script has three functions. It renders to the sceen or "paints"
a blank HTML form prompting for a search term if none is given. If
a search term is given, the script looks through our DIT and return
the results to the generic paint function. If a entry is edited, it
makes the requested change and repaints the screen.
Lines 26 to 49 contain an odd
The first interesting function begins on line 83. RFC 2254 describes the many ways that LDAP can compare data. In
Figure 9, we use only one of those ways: the partial case insensitive
match. If the search term matches any part of the country, email, full
name, or telephone fields, the search is considered successful. More
refined search functions can easily be created.
LDAP requires a somewhat odd syntax for to describe search filters.
Like anything the deals with a set of data, LDAP defines a group of
boolean operators (e.g. "or" and "and"). These must precede the terms,
which are just attribute name-value pairs, that they join.
Fortunately, the operators themselves should look familiar:
In the first example, we are looking for entries in the DIT that have
a country field of The On line 120, the search terms are passed to This Entry object is our interface to an individual DIT entry. Because
all fields in an entry can be mutli-valued, the Spreadsheets are a useful metaphors for manipulating tabular data,
such as this address book. Implementing this metaphor calls for some
tricky HTML code. Lines 148 to 164 create one row representing one DIT
entry. This row is an editable form that can update the entry if the
user changes any values. Figure ??? shows the result of searching for an
empty string. This is a special case in this application which returns
the whole DIT. The number of entries in this address book was trimmed
for this screenshot, but I specifically left my mother's name in the
list to give her some reward for reading this article. She's not really
into Perl and doesn't quite know what I do. (Mom, this is what I do.)
Modifying an existing DIT node requires first locating the desired
entry. This code locates entries by searching for the right If the user cleared out the cn field in the form, we will erase that
entry. Otherwise, the script will call the Entry object's replace
method to change the relevant fields. The Entry does not get updated
on the LDAP server until the update method is called on line 200.
Figure 3 shows the result of looking for
So there we have it. In about two hundred lines of code, we have a
platform-independent address book. You can easily adapt the concept
shown here to make a fabulous Perl/Tk version. One important note:
this code does not ensure data integrity on updates. That is, if
multiple users attempt to update the DIT at the same time, LDAP will
make no attempt to lock its data. This sort of "Atomic Consistency
Isolation and Durability" (ACID) support is well beyond LDAP's
capabilities. If you find yourself needing it, use a real relational
database management system.
LDAP is becoming pervasive. It forms the backbone of both Microsoft's
Active Directory system as well as Novell's Network Directory
Services. Netscape has also been very active in developing their own
LDAP server implementation, and even Sendmail, Inc. is supporting LDAP
address systems. You'll likely see directory services mature and grow,
eventually eclipsing such old network standards like NIS and possibly
DNS.
__END__
Joe Johnston ( jjohn@oreilly.com ) has just discovered the joy and
the pain of Net::LDAP object that expects to find an
LDAP server on a machine called daisypark.net. Recall that we can
make changes, like adding entries, to the DIT only if we are the
authorized user "Manager". We use the Net::LDAP bind method to
"login" to the LDAP server as this account. We pass the password
and the full Icn=Manager, dc=daisypark, dc=net, to the bind method to
prove that we are who we say we are.
add method. Line 25 does
this with the parameters dn and attr. We put the attributes of
the entry in an anonymous array of name-value pairs. To determine
whether the add fails, we must examine the object returned by this
method. This Net::LDAP::Message object, labeled in our script as
$results, has a method called code. If this method returns a
non-zero value, an error occurred during the add call.
unbind method.
ldapsearch.
We can also use the Netscape 4.x Mail address book, itself
an LDAP client, to look at our LDAP server. Either way, we can
perform various searches until we are satisfied all went well
with the data loading.
Net::LDAP module.
Let's build a searchable, editable web client interface to this DIT.
A Searchable Web Interface To Manage Your Directory
CGI, CGI::Carp,
CGI::Pretty, and Net::LDAP. Efficiency wonks will note that
the nicely formatted HTML output of CGI::Pretty it both slow and
unneccessary. However, it's nice being able to look at human
readable HTML during development. Also great for debugging
is the fatalsToBrowser option of CGI::Carp.
for loop. This is simply a switch
statement that enumerates the three functions of the program. The
script looks for the CGI variable "action" to determine which function
to execute. The default function is to paint a blank form.
1 #!/usr/bin/perl --
2 # jjohn 6/2000
3 # A CGI interface to a LDAP server.
4
5 use strict;
6 use CGI qw/:all *table/;
7 use CGI::Carp qw/fatalsToBrowser/;
8 use CGI::Pretty qw/:all/;
9 use Net::LDAP;
10
11 # first, set up the main objects
12 my $cgi = CGI->new();
13
14 my $base_dn = 'dc=daisypark, dc=net';
15 my $conn = Net::LDAP->new("ldap.daisypark.net") or
16 die "ERROR: Can't connect: $@";
17
18 # Make a authenticated connection with the s3cr3t password
19 $conn->bind( dn => "cn=Manager, $base_dn",
20 password => 's3cr3t',
21 );
22
23 # Have we been asked to do anything?
24 my $action = $cgi->param('action');
25
26 for ($action){
27 my $message = '';
28 /search/ && do {
29 $message = search(
30 ldap => $conn,
31 base_dn => $base_dn,
32 cgi => $cgi,
33 );
34 };
35
36 /modify/ && do {
37 $message = modify(
38 ldap => $conn,
39 base_dn => $base_dn,
40 cgi => $cgi,
41 );
42 };
43
44 paint( ldap => $conn,
45 base_dn => $base_dn,
46 cgi => $cgi,
47 message => $message,
48 );
49 }
50
51 $conn->unbind;
52 exit;
53
54 #-------
55 # Subroutines
56 #-------
57 sub paint{
58 my %params = @_;
59
60 print
61 header,
62 start_html( -bgcolor => "#FFFFFF",
63 -title => ($params{title} ||
64 'View LDAP for Daisypark'),
65 ),
66 h2( ($params{title} || 'View LDAP for Daisypark') ),
67 hr,
68 ($params{message} || 'Please perform a search');
69
70 print
71 hr,
72 start_form,
73 '<INPUT TYPE="HIDDEN" NAME="action" VALUE="search">',
74 'Search: ',
75 textfield( -name => 'search'),
76 submit,
77 end_form,
78 end_html;
79 }
80
81 # ldap_lookup() returns an LDAP entry object
82 # for the provided search term.
83 sub ldap_lookup {
84 my %params = @_;
85
86 my $criteria = $params{search};
87 my $filter;
88
89 # hack, I want to search for everything
90 # if I have an empty string
91 undef $criteria if $criteria eq '';
92
93 for (qw/c mail sn cn telephonenumber/) {
94 # todo: $params{search} needs to escape meta-chars!
95 if ( defined $criteria ) {
96 $filter .= "($_=*".$criteria."*) ";
97 } else {
98 $filter .= "($_=*) ";
99 }
100 }
101 $filter = "(| $filter)";
102
103 my $mesg = $params{ldap}->search(
104 base => $params{base_dn},
105 filter => $filter,
106 );
107 if ( $mesg->code ) {
108 die "Oops ($filter): ", $mesg->error;
109 }
110
111 return $mesg;
112 }
113
114 # search() performs a lookup in the LDAP for a given string,
115 # returning a nice HTML table.
116
117 sub search {
118 my %params = @_;
119
120 my $mesg = ldap_lookup( @_,
121 search => ( $params{search} ||
122 $params{cgi}->param('search')
123 ),
124 );
125
126 if ( $mesg->count == 0 ) {
127 return "No matches found for '$params{search}'";
128 }
129 my $results;
130 $results .= p(small('Matches: ' .
131 b($mesg->count) .
132 ' for term ' .
133 b( $params{cgi}->param('search') )
134 ));
135 $results .= start_table( -cellspacing => 0,
136 -cellpadding => 0,
137 );
138 $results .= Tr(
139 th({-bgcolor=>'pink'},
140 [qw/Name E-Mail Phone Change/])
141 );
142 # add some pretty color every third row
143 my $row = 0;
144 for my $entry ( $mesg->all_entries ) {
145
146 my $cn = $entry->get('cn')->[0];
147
148 $results .= Tr(
149 {-bgcolor => (!($row%3) ? "#CCCCCC" :"#FFFFFF") },
150 start_form,
151 '<INPUT TYPE="HIDDEN" NAME="action" VALUE="modify">',
152 qq/<INPUT TYPE="HIDDEN" NAME="old_cn" VALUE="$cn">/,
153 td(textfield( { -name => 'cn',
154 -default => $entry->get('cn')
155 })),
156 td(textfield( {-name => 'mail',
157 -default => $entry->get('mail')
158 })),
159 td(textfield( {-name => 'telephonenumber',
160 -default => $entry->get('telephonenumber')
161 })),
162 td( submit ),
163 end_form,
164 );
165
166 $row++;
167 }
168
169 return $results .= end_table;
170 }
171
172 sub modify {
173 my %params = @_;
174
175 my $old_cn = $params{cgi}->param('old_cn');
176 my $mesg = ldap_lookup( @_,
177 search => $old_cn );
178
179 if ( $mesg->count == 0 ) {
180 return "Oops: Can't find $old_cn";
181 }
182
183 my $cgi = $params{cgi};
184 my $entry = $mesg->entry(0); # really need to iterate over results
185
186 # Delete if 'cn' is empty, else modify
187 my $report = '';
188 if ( $cgi->param('cn') =~ /^\s*$/ ) {
189 $entry->delete();
190 $report = "Deleted";
191 } else {
192 $entry->replace(
193 cn => $cgi->param('cn'),
194 mail => $cgi->param('mail'),
195 telephoneNumber => $cgi->param('telephonenumber'),
196 );
197 $report = "Updated";
198 }
199
200 $entry->update( $params{ldap} );
201 return $report . " " . $cgi->param('cn');
202 }
Listing 9 ldap_lookup()
queries the LDAP server for the given terms and returns the results as
an LDAP message object.
Figure 1: Web client awaiting search term
(| (c=US)(cn=joe*) )
(& (c=US)(cn=joe*) )
Listing 10 US or have canonical names that begin with joe.
In the second example, we have the same terms "and"ed together, which
selects only entries with both of those criteria.
Net::LDAP search method returns a message object. We can check for
error conditions by looking at the numeric code returned from the
aptly named code() method. Any non-zero value indicates an error. The
particular error message can be retrieved with a call to the error()
method.
ldap_lookup(). Assuming
nothing fatal has happened, we then look at the message object's
count() method, which returns the number of matched entries for our
terms. Provided at least one entry matches, we can use a simple
foreach loop to iterate over all the entry objects returned by the
message object's all_entries() method.
get() method returns an
anonymous list of values.
Figure 2: Web client displaying the whole DIT cn
field. This value was stored in a hidden field before the user made
changes to this record. Here, I'll admit to a fudge. The search could
return more than one entry. After all, cn is not exactly the same as
the DN. There are no guarantees that the cn will be unique. A better
value to have stored away, of course, is the DN of the desired
entry. That's what it's there for.
bill. I'll add his email
address and press the submit button. Figure 4 shows the dramatic
results.
Figure 3: Entry before modification
Figure 4: Entry after modification Where LDAP Is Going
gnapster and has learned that just because he can
download the entire Olivia Newton-John catalog doesn't mean he
should. He created the Aliens, Aliens, Aliens web site
( http://aliensaliensaliens.com ).