Kelly Yeoh

LDB part 1 (of god only knows how many!!!) – Introduction

• November 2, 2011 12:37 am

LDB Posts

• *** LDB part 1 (of god only knows how many!!!) – Introduction
• LDB part 2 (it just keeps goin’) – Tools and things

 

Introduction

LDB is the database engine used within Samba. It is a light-weight, LDAP-*like* database, but not completely LDAP compliant.

Raw LDB is not even close to LDAP compliant, we use modules to ensure compliance. We aim for LDAP compliance where possible, although Samba’s higher priority is Active Directory compliance. We do aim for LDAP compliance where it doesn’t conflict.

LDB is transactional (allowing multiple changes to be made to ensure that changes all are applied as expected prior to committing them to the database, if an error occurs all changes are backed out and the database is left “untouched”) and modular (allowing different information or functionality to be added or removed according to how a database is needed to perform).

The main LDB website is located at http://ldb.samba.org

All of the examples listed below will be using the ldb databases created by the self test suite included with Samba. These databases are located at {samba_build_directory}/st/dc/private/

 

What is LDAP?

Lightweight Directory Access Protocol is the dominant database used by organisations around the world to manage their, well, organisation – eg all of their directory services (things such as person information, IT information (machine details, print services, etc), all information pertaining to the business that would need to be “directoried”). Many large companies use LDAP within their server and business tools, eg IBM, Oracle, HP, Novell to name but a few.

LDAP is an incredibly complex protocol, and there are many books and RFCs available to read about it. The RFC probably most of note is this one: RFC4511 – Lightweight Directory Access Protocol (LDAP): The Protocol

Since this post aims to address LDB, I’ll go further into LDAP and how we use it at a later date. For now, some basic LDAP information (and some nice links) are available here: http://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol

 

Active Directory

Active Directory is a Microsoft product, based around LDAP, but uses other pieces to make up the whole such as Kerberosv5, DNS, MS-RPC, SMB (CIFS). Other protocols are used within Active Directory, but these form the major components.

These components are used as follows:
LDAP database
kerberos authentication system
dns machine names – not as trivial as may be expected, this forms a fairly large part of AD and will probably warrant its own post in the near future. If you want this, and I forget, ping me!! Really
MS-RPC remote procedure calls, if you can’t do **something** over LDAP you can generally use MS-RPC. With MS-RPC, you create a protocol specification using IDL files, compiled using IDL compilers, which creates a remote protocol procedure for RPC calls. This also warrants its own post to demonstrate why, how, where, etc. Soon!
SMB/CIFS for file sharing

There are several sources of information available on Active Directory. Microsoft has many useful documents available here:
http://msdn.microsoft.com/en-us/library/gg258393%28v=PROT.13%29.aspx

And probably the one of most use, particularly within the Samba team, is the Active Directory Technical Specification. For example, this document gives valuable information about the database schema which is useful for ensuring compatibility with data sources within Samba (see section 3.1.1.3.1.1 Schema, page 123 of 521, using version 30.0 of the document, released 23rd September 2011)
http://msdn.microsoft.com/en-us/library/cc223122%28v=PROT.13%29.aspx

 

TDB

TDB is the Trivial DataBase. This is a very simple key/value pair database (where each record has a key and some data associated with that key). TDB stores data in binary format, and supports transactions.

The main TDB website is located at http://tdb.samba.org

Some basic operations

tdb_open Opens a database
tdb_close Closes a database
tdb_delete Deletes a record with a given key
tdb_exists Checks for existence of a record with a given key
tdb_fetch Retrieves a record with a given key
tdb_store Allows insertion, modification or replacement of a record

TDB also has other functionality such as enumerating all records by asking for a list of all keys, start a transaction, do any number of operations and then cancel or commit transaction, etc.

Some typical tdb data might look as follows (taken from the samba secrets.ldb database used with the self test suite). This clearly displays the key and value pair of each record, with each record delimited by curly braces (in a dump format):

username@computername:~/Samba/samba-master$ tdbdump st/dc/private/secrets.ldb
{
key(12) = "DN=@MODULES\00"
data(45) = "g\19\01&\01\00\00\00@MODULES\00@LIST\00\01\00\00\00\0D\00\00\00samba_secrets\00"
}
{
key(43) = "DN=FLATNAME=SAMBADOMAIN,CN=PRIMARY DOMAINS\00"
data(702) = "g\19\01&\11\00\00\00flatname=SAMBADOMAIN,cn=Primary Domains\00msDS-KeyVersionNumber\00\01\00\00\00\01\00\00\001\00objectClass\00\03\00\00\00\03\00\00\00top\00\0D\00\00\00primaryDomain\00\0E\00\00\00kerberosSecret\00objectSid\00\01\00\00\00\18\00\00\00\01\04\00\00\00\00\00\05\15\00\00\00\013\BE\F5\DB\E8\C33\B4\9C\1E\ED\00privateKeytab\00\01\00\00\00\0E\00\00\00secrets.keytab\00realm\00\01\00\00\00\11\00\00\00SAMBA.EXAMPLE.COM\00saltPrincipal\00\01\00\00\000\00\00\00host/localdc.samba.example.com@SAMBA.EXAMPLE.COM\00samAccountName\00\01\00\00\00\08\00\00\00LOCALDC$\00secret\00\01\00\00\00\11\00\00\00machinelocDCpass1\00secureChannelType\00\01\00\00\00\01\00\00\006\00servicePrincipalName\00\02\00\00\00\0C\00\00\00HOST/localdc\00\1E\00\00\00HOST/localdc.samba.example.com\00objectGUID\00\01\00\00\00\10\00\00\00H\D9_\D5+\E3\C0G\B7\CF\F6\FB\B1m\EB\1E\00whenCreated\00\01\00\00\00\11\00\00\0020111005000140.0Z\00whenChanged\00\01\00\00\00\11\00\00\0020111005000140.0Z\00uSNCreated\00\01\00\00\00\01\00\00\007\00uSNChanged\00\01\00\00\00\01\00\00\007\00name\00\01\00\00\00\0B\00\00\00SAMBADOMAIN\00flatname\00\01\00\00\00\0B\00\00\00SAMBADOMAIN\00"
}
{
key(31) = "DN=@INDEX:FLATNAME:SAMBADOMAIN\00"
data(111) = "g\19\01&\02\00\00\00@INDEX:FLATNAME:SAMBADOMAIN\00@IDXVERSION\00\01\00\00\00\01\00\00\002\00@IDX\00\01\00\00\00'\00\00\00flatname=SAMBADOMAIN,cn=Primary Domains\00"
}
{
key(15) = "DN=@ATTRIBUTES\00"
data(153) = "g\19\01&\04\00\00\00@ATTRIBUTES\00cn\00\01\00\00\00\10\00\00\00CASE_INSENSITIVE\00flatname\00\01\00\00\00\10\00\00\00CASE_INSENSITIVE\00realm\00\01\00\00\00\10\00\00\00CASE_INSENSITIVE\00sAMAccountName\00\01\00\00\00\10\00\00\00CASE_INSENSITIVE\00"
}

Note: this contains only a small section of the data returned from the secrets.ldb database.

Within Samba, the key is always the distinguished name (DN) for an object, and the data contains all of the attributes for that object. Every object in LDAP has a distinguished name that distinguishes it from any other (unique, as the name would suggest ).

 

How LDB uses TDB

LDB basically sits on top of TDB and manipulates the data into an LDAP-like structure. As a very basic example, we can imagine a stack like structure with TDB residing at the base, then LDB modules are stacked on top that describe the functionality or rules required for this database (for example, schema information), and the LDB API sits on top taking and servicing requests.

So where TDB uses a key/value pair that is quite confusing to look at:

{
key(43) = "DN=FLATNAME=SAMBADOMAIN,CN=PRIMARY DOMAINS\00"
data(702) = "g\19\01&\11\00\00\00flatname=SAMBADOMAIN,cn=Primary Domains\00msDS-KeyVersionNumber\00\01\00\00\00\01\00\00\001\00objectClass\00\03\00\00\00\03\00\00\00top\00\0D\00\00\00primaryDomain\00\0E\00\00\00kerberosSecret\00objectSid\00\01\00\00\00\18\00\00\00\01\04\00\00\00\00\00\05\15\00\00\00\013\BE\F5\DB\E8\C33\B4\9C\1E\ED\00privateKeytab\00\01\00\00\00\0E\00\00\00secrets.keytab\00realm\00\01\00\00\00\11\00\00\00SAMBA.EXAMPLE.COM\00saltPrincipal\00\01\00\00\000\00\00\00host/localdc.samba.example.com@SAMBA.EXAMPLE.COM\00samAccountName\00\01\00\00\00\08\00\00\00LOCALDC$\00secret\00\01\00\00\00\11\00\00\00machinelocDCpass1\00secureChannelType\00\01\00\00\00\01\00\00\006\00servicePrincipalName\00\02\00\00\00\0C\00\00\00HOST/localdc\00\1E\00\00\00HOST/localdc.samba.example.com\00objectGUID\00\01\00\00\00\10\00\00\00H\D9_\D5+\E3\C0G\B7\CF\F6\FB\B1m\EB\1E\00whenCreated\00\01\00\00\00\11\00\00\0020111005000140.0Z\00whenChanged\00\01\00\00\00\11\00\00\0020111005000140.0Z\00uSNCreated\00\01\00\00\00\01\00\00\007\00uSNChanged\00\01\00\00\00\01\00\00\007\00name\00\01\00\00\00\0B\00\00\00SAMBADOMAIN\00flatname\00\01\00\00\00\0B\00\00\00SAMBADOMAIN\00"
}

LDB would output the same data as follows, in a much more readable fashion:

dn: flatname=SAMBADOMAIN,cn=Primary Domains
msDS-KeyVersionNumber: 1
objectClass: top
objectClass: primaryDomain
objectClass: kerberosSecret
objectSid: S-1-5-21-4122882817-868477147-3978206388
privateKeytab: secrets.keytab
realm: SAMBA.EXAMPLE.COM
saltPrincipal: host/localdc.samba.example.com@SAMBA.EXAMPLE.COM
samAccountName: LOCALDC$
secret: machinelocDCpass1
secureChannelType: 6
servicePrincipalName: HOST/localdc
servicePrincipalName: HOST/localdc.samba.example.com
objectGUID: d55fd948-e32b-47c0-b7cf-f6fbb16deb1e
whenCreated: 20111005000140.0Z
whenChanged: 20111005000140.0Z
uSNCreated: 7
uSNChanged: 7
name: SAMBADOMAIN
flatname: SAMBADOMAIN
distinguishedName: flatname=SAMBADOMAIN,cn=Primary Domains

 

LDB Modules

Modules can be stacked on top of each other to add rules or functionality. As an example a schema can be added as a module to a database to ensure that the data therein complies with set rules (so a database can be forced to be entirely LDAP compliant, as an example).

Another example of a module used with LDB is the password hash module. This sits between the database and the LDB API and ensures that passwords match password policies, and also performs any required encryption/decryption thereof.

 

More on Schemas

Raw LDB is schema-less, but it is designed to allow modules to enforce schema and add functionality. To begin with, there are no rules and regulations, and the LDB API allows for fast and easy setup of a database with minimal overheads. Further overheads can be added as required using modules (as mentioned above), and schema support can be added in this way.

It is through these schemas that we ensure that our databases are LDAP compliant. In this way, we can use LDB within Samba to allow us to communicate with other LDAP databases, and (what we are really aiming for) Active Directory.

Schema-less

an example of a schema-less database used within Samba is the secrets.ldb database. The reason that secrets.ldb doesn’t need a schema is because it is used only internally within Samba, so there is no need for the overhead of a schema. Because there is no schema attached, arbitrary data can be inserted into the database, for example we can modify a record to include a new attribute called “pet” with a value of our choosing, eg “cat”, and this is a perfectly acceptable thing to do. It is also possible that I have a thing for cats

# editing 1 records
# record 1

dn: samAccountName=dns-localdc,CN=Principals
msDS-KeyVersionNumber: 1
objectClass: top
objectClass: secret
objectClass: kerberosSecret
privateKeytab: dns.keytab
realm: SAMBA.EXAMPLE.COM
sAMAccountName: dns-localdc
secret: DZCb)wH]7uoDEd:q+ZmoC#P==0YAP(7.Dbz$6RA0l!KLx3sr#0]oV>eij3,=y@StOKNCn?k28ik%McPo&tPLnoyt0koqxm@NFe(aRn;YyD4fIkK4w>K=zBjUkv_LEGUCcoK_:7Q#0UC2CZx0WOV6pHu$aO
servicePrincipalName: DNS/samba.example.com
servicePrincipalName: DNS/localdc.samba.example.com
objectGUID: 2f1d38f3-49b2-4baa-b77c-14c5daebadc0
whenCreated: 20111005000141.0Z
uSNCreated: 8
name: dns-localdc
pet: cat
whenChanged: 20111005004406.0Z
uSNChanged: 9
distinguishedName: samAccountName=dns-localdc,CN=Principals

(note the sneaky inclusion of the new attribute just after the name attribute. Also note the plain text way in which the data has been written, this is called LDIF – LDAP Data Interchange Format. More on this in a later post)

Schema controlled

We can add a schema module to a database to ensure that it follows set rules. A schema would generally include information about what attributes are allowed, and constraints on data type, size, etc. Within Samba, the sam.ldb database is controlled by a schema. Schemas are only needed where the database communicates externally, so a schema provides valuable protection to ensure data compatibility. If we were to make an attempt to include the same pet: cat attribute and value pair within the sam.ldb database, the following error would be returned:

Controllers,DC=samba,DC=example,DC=com - objectclass_attrs: attribute 'pet' on entry 'CN=LOCALDC,OU=Domain Controllers,DC=samba,DC=example,DC=com' was not found in the schema!

The schema for the sam.ldb database is compatible with the schema described within the Microsoft Active Directory Technical Specification http://msdn.microsoft.com/en-us/library/cc223122%28v=PROT.13%29.aspx

The schemas used are available for viewing online via the http accessible copy of Samba code.

http://samba.org/ftp/unpacked/samba_4_0_test/source4/setup/ad-schema/MS-AD_Schema_2K8_R2_Attributes.txt defines the attributes regardless of objectClass. The classes file determines which attributes are associated with each class

http://samba.org/ftp/unpacked/samba_4_0_test/source4/setup/ad-schema/MS-AD_Schema_2K8_R2_Classes.txt defines the objectClass and what attributes are allowed for a given class

 

coming up:

• LDB tools (and examples)
• Hidden attributes
• How LDB indexing works
• rootDSE

• Filed under
  • LDB
  • , Samba
• Add your comment
• TrackBack URI
• Comments feed