I recently ran into a situation at a customer involving a self-signed certificate and an LDAPS connection to a domain controller. My first instinct was: “Nooooo, that’s not the way!” Probably a familiar reaction for anyone working with PKI and a love for Star Wars.

But as a temporary solution, it can serve a purpose. Would I recommend it as a production design? No. Is it easy to manage as the environment grows? Also no. Does it work? Absolutely, albeit with a few catches.

If you have a single external application or service connecting to a specific domain controller, the setup is relatively straightforward, create a suitable certificate and explicitly configure the application to trust it. Another application connecting to that same DC will also need to trust the certificate. Without that trust, certificate validation should fail.

Things become more complicated when the application connects to the domain name, such as corp.michaelwaterman.nl, instead of a specific DC. Many third-party applications simply resolve that name through DNS rather than using the Windows DC Locator process. Depending on the DNS configuration, they may reach different domain controllers.

Each DC that the application might reach therefore needs a suitable LDAPS certificate, and the application must trust every one of those self-signed certificates. Each certificate must also cover the name the application uses to connect. Renewing a certificate means updating that trust again.

That is where a temporary workaround starts becoming a maintenance exercise. Still, for a narrowly scoped emergency, it can get you moving. Let’s look at how to configure it, and verify that it actually works.

First, give the DC something it can use

For this example, I’m using the following lab configuration:

SettingValue
Domain controllerLAB-DC-01.corp.michaelwaterman.nl
Domaincorp.michaelwaterman.nl
IP address192.168.66.3

The certificate needs the Server Authentication EKU (1.3.6.1.5.5.7.3.1), a private key that AD DS can access, and the DC’s FQDN in the subject or a DNS SAN. I’m putting the FQDN in both the CN and SAN. I’m also adding the domain name and IP address to the SAN so we can test those connection names.

Adding the domain name does not automatically make the certificate suitable for every DC in the domain. This certificate and its private key belong to this particular DC. Remember that.

There’s another small detail here: an IP address needs to be encoded as an IP Address SAN. Putting 192.168.66.3 in a DNS SAN might look convincing when you read it, but it is a different type of entry. Certificate validation knows the difference, you should too.

Let’s create the certificate

Run the following PowerShell as administrator on the domain controller. The script creates a certificate valid for 30 days, keeps the private key non-exportable, installs the public certificate in the DC’s trusted root store, and exports a .cer file to the desktop.

$dcFqdn = 'LAB-DC-01.corp.michaelwaterman.nl'
$domain = 'corp.michaelwaterman.nl'
$ip     = '192.168.66.3'

$cert = New-SelfSignedCertificate `
    -Type Custom `
    -Subject "CN=$dcFqdn" `
    -TextExtension @(
        '2.5.29.37={text}1.3.6.1.5.5.7.3.1'
        "2.5.29.17={text}DNS=$dcFqdn&DNS=$domain&IPAddress=$ip"
    ) `
    -KeyUsage DigitalSignature, KeyEncipherment `
    -CertStoreLocation 'Cert:\LocalMachine\My' `
    -Provider 'Microsoft RSA SChannel Cryptographic Provider' `
    -KeyAlgorithm RSA `
    -KeyLength 3072 `
    -KeySpec KeyExchange `
    -HashAlgorithm SHA256 `
    -KeyExportPolicy NonExportable `
    -NotAfter (Get-Date).AddDays(30) `
    -FriendlyName 'Temporary LDAPS - DNS and IP - Server Auth Only' `
    -ErrorAction Stop

$exportPath = Join-Path `
    ([Environment]::GetFolderPath('Desktop')) 'Temporary-LDAPS-IP.cer'

Export-Certificate `
    -Cert $cert `
    -FilePath $exportPath `
    -Force `
    -ErrorAction Stop | Out-Null

Import-Certificate `
    -FilePath $exportPath `
    -CertStoreLocation 'Cert:\LocalMachine\Root' `
    -ErrorAction Stop | Out-Null

# Controleer de EKU
$cert | Format-List Subject, EnhancedKeyUsageList, Thumbprint

# Controleer de SAN
$cert.Extensions |
    Where-Object { $_.Oid.Value -eq '2.5.29.17' } |
    ForEach-Object { $_.Format($true) }

Note! set -KeyExportPolicy Exportable, if you like to export the certificate afterwards and put it into the NTDS sore, like I explained in another blog post.

You might wonder, Why -Type Custom? During this exercise, the certificate created with -Type SSLServerAuthentication contained both Server Authentication and Client Authentication. LDAPS does not need the latter, so I explicitly define the EKU. The expected result is:

Subject : CN=LAB-DC-01.corp.michaelwaterman.nl
EnhancedKeyUsageList : {Server Authentication (1.3.6.1.5.5.7.3.1)}
HasPrivateKey : True

The SAN extension now contains:

DNS Name=LAB-DC-01.corp.michaelwaterman.nl
DNS Name=corp.michaelwaterman.nl
IP Address=192.168.66.3

Check that last line. We want IP Address, not DNS Name followed by an IP address.

A certificate in the root store is only half the story

The script puts the certificate in two stores on the DC:

Certificate storeWhat goes there
Local Computer → PersonalCertificate with its private key
Local Computer → Trusted Root Certification AuthoritiesPublic certificate used to establish trust

The DC needs the private key to prove its identity during the TLS handshake. Importing only the exported .cer into Trusted Root does not give it that capability. The application connecting to the DC also needs to trust the public certificate. The private key stays on the DC. Please leave it there, no seriously don’t export it. For a Windows client, you can import the .cer as administrator:

Import-Certificate `
    -FilePath 'C:\Temp\Temporary-LDAPS-IP.cer' `
    -CertStoreLocation 'Cert:\LocalMachine\Root'

If you need to distribute it to several Windows computers, you can use a computer GPO:

Computer Configuration → Policies → Windows Settings → Security Settings → Public Key Policies → Trusted Root Certification Authorities

Import the public certificate there and apply the GPO to the relevant computers. For an application with its own truststore, configure trust there. Importing a certificate on an Ubuntu host, is a whole different story. But no worries, this blog got you covered for that as well, just keep on reading.

Now let AD DS load it

There are ways to refresh the LDAPS certificate without restarting the entire server, but a restart was convenient for this test. In a customer environment, plan that appropriately.

Also, keep track of existing certificates. If you create several suitable certificates while testing, do not assume AD DS will present the one you created last. Check the certificate it actually offers. A certificate in the NTDS service store can also take precedence over certificates in the local computer store. See my blog on the subject here.

What if it doesn’t work?

One of my first ldaps test attempt returned this:

no peer certificate available
SSL handshake has read 0 bytes
write:errno=104

Schannel on the DC added:

No suitable default server credential exists on this system.

Credentials? Do I need to provide a username and password? No. In this message, the server credential is the certificate and its associated private key. LDAP authentication comes later, after TLS has been established. The original certificate had an IP address as its CN and did not contain the DC’s own FQDN. The DC had not been restarted either.

After replacing the certificate with one containing the FQDN and reloading AD DS, the connection worked. See I make mistakes as much as the next person. Since I changed more than one thing, I cannot honestly point at one of them and declare it the sole cause. Troubleshooting would be a lot easier if we always changed one variable at a time. Apparently, I needed that reminder too.

Testing from Windows

For a quick check, open ldp.exe and select Connection → Connect. Enter:

  • The DC’s FQDN.
  • Port 636.
  • Enable SSL.

I tested both the FQDN and IP address successfully. Great! But I also wanted to test from Ubuntu and explicitly check the certificate’s identity. A successful connection alone does not tell us exactly which validation checks a client performed.

Let’s bring in OpenSSL

Copy Temporary-LDAPS-IP.cer from the DC to Ubuntu (or whatever Linux flavor you prefer). The PowerShell export produces a DER-encoded certificate. Convert it to PEM:

openssl x509 \
  -inform DER \
  -in Temporary-LDAPS-IP.cer \
  -out Temporary-LDAPS-IP.pem

Set the connection details:

DC_FQDN='LAB-DC-01.corp.michaelwaterman.nl'
DC_IP='192.168.66.3'

For these tests, I’m using -CAfile to explicitly trust the exported self-signed certificate. There is no need to install it in Ubuntu’s system trust store. If you want Ubuntu to trust it system-wide, copy the PEM file to /usr/local/share/ca-certificates/ using a .pem extension, then update the trust store:

sudo cp Temporary-LDAPS-IP.pem \
  /usr/local/share/ca-certificates/Temporary-LDAPS-IP.pem

sudo update-ca-certificates

You can then repeat the OpenSSL tests without -CAfile. Applications with their own truststore, including applications running in containers, may still need separate configuration.

Test the FQDN

Let’s start with the DC’s FQDN. We’ll check whether the TLS connection works, the certificate is trusted, and its name matches the hostname we’re connecting to.

openssl s_client \
  -connect "${DC_FQDN}:636" \
  -servername "${DC_FQDN}" \
  -CAfile Temporary-LDAPS-IP.pem \
  -verify_hostname "${DC_FQDN}" \
  -purpose sslserver \
  -verify_return_error \
  -showcerts </dev/null

A few options deserve attention:

OptionWhat it does
-connectSelects the destination and port
-servernameSends the DNS name as TLS SNI
-CAfileSupplies the certificate trusted for this test
-verify_hostnameChecks the certificate against the expected DNS name
-purpose sslserverChecks suitability for TLS server authentication
-verify_return_errorAborts the handshake if certificate verification fails

Specifying a destination with -connect does not automatically enable hostname validation. That is why -verify_hostname is included explicitly. Sending SNI with -servername is not a substitute for that check either.

Test the IP address

Now the same idea, using the IP address:

openssl s_client \
  -connect "${DC_IP}:636" \
  -noservername \
  -CAfile Temporary-LDAPS-IP.pem \
  -verify_ip "${DC_IP}" \
  -purpose sslserver \
  -verify_return_error \
  -showcerts </dev/null

Here, -verify_ip checks the certificate’s IP Address SAN. The CN can remain the DC’s FQDN. Both tests returned:

Verification: OK
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Verify return code: 0 (ok)

Tha thaaaa! Both connection names worked, including explicit identity validation.

A good test should also fail

So far, we have asked OpenSSL to validate names that are present in the certificate. Let’s give it a name that should fail. The destination remains the real DC. Only the identity we ask OpenSSL to verify changes:

openssl s_client \
  -connect "${DC_IP}:636" \
  -noservername \
  -CAfile Temporary-LDAPS-IP.pem \
  -verify_ip '192.0.2.123' \
  -verify_return_error </dev/null

The result:

verify error:num=64:IP address mismatch
certificate verify failed
Verify return code: 64 (IP address mismatch)

Excellent. A failure we actually wanted, which sounds…. weird.

TestResult
Correct FQDNAccepted
Correct IP Address SANAccepted
Incorrect IP addressRejected

One interesting detail: the failed test also printed:

no peer certificate available

That looked familiar, during my initial testing I received the same error message, but the output tells a different story this time. OpenSSL received the server certificate, checked it, and rejected the IP address. The handshake had read 1812 bytes, compared with 0 bytes during the original failure. The same line in the output does not always mean the same underlying problem. Context matters.

The LDAP connectivity test

OpenSSL confirmed that TLS and certificate validation work. Great! Now let’s actually authenticate and run an LDAP query from Ubuntu. First, install the LDAP client tools:

sudo apt install ldap-utils

From the directory containing the PEM certificate, run:

LDAPTLS_CACERT="$PWD/Temporary-LDAPS-IP.pem" \
LDAPTLS_REQCERT=demand \
ldapsearch \
  -x \
  -H 'ldaps://LAB-DC-01.corp.michaelwaterman.nl:636' \
  -D 'superuser@corp.michaelwaterman.nl' \
  -W \
  -b 'DC=corp,DC=michaelwaterman,DC=nl' \
  -s sub \
  -z 10 \
  '(&(objectCategory=person)(objectClass=user))' \
  sAMAccountName displayName userPrincipalName

The -W option prompts for the password, so it stays out of the command and your shell history. Although -x selects simple authentication, the credentials travel inside the encrypted LDAPS connection. LDAPTLS_REQCERT=demand requires successful server certificate validation.

The query searches the domain for user objects and returns their account name, display name, and UPN, with a maximum of ten entries. If you see Size limit exceeded after receiving results, that can simply mean the query reached our ten-entry limit. If the query returns users, we’ve now tested the whole path: establishing TLS, validating the server certificate, authenticating to AD, and retrieving directory information. That’s a little more convincing than an open port 636.

And when the certificate expires?

This is where the temporary solution needs some attention again. A replacement self-signed certificate must be distributed to the clients that need to trust it. Reusing the same subject and SAN entries does not make it the same trusted certificate.

For this example, that means updating the Windows GPO or application truststore, replacing the PEM file used for testing, and checking that AD DS presents the replacement certificate. Also, these OpenSSL tests validate the TLS connection. They do not perform an LDAP bind or prove that an application’s LDAP configuration works. That is the next layer to test.

Would I still prefer a CA-issued certificate? Absolutely. But if you need a temporary LDAPS connection, this gives you a working setup, and a way to verify that it accepts the right identity and rejects the wrong one.

That’s it for today, until next time!

Resources

Enable Lightweight Directory Access Protocol (LDAP) over Secure Sockets Layer (SSL) – Windows Server | Microsoft Learn

[MS-ADTS]: renewServerCertificate | Microsoft Learn

Event ID 1220 — LDAP over SSL | Microsoft Learn