Skip to content
View in the app

A better way to browse. Learn more.

Gear Crushers

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

SSL Troubleshooting on the F5

Featured Replies

SSL_flow2_3_26_15.png

ClientHello

When a client first attempts to connect to an SSL server, it initiates the session by sending a ClientHello message to the server. The ClientHello message starts the SSL communication between the two systems. The ClientHello message contains some of the following components:

Version: The version field contains the highest SSL version that the client supports.

Random: A random number generated by the client.

Session ID: An arbitrary sequence of bytes chosen by the server; it identifies a particular SSL session. The client may attempt to resume a previously established session by sending a non-zero session ID.

Cipher suites: Identifies the list of ciphers suites that the client supports.

Compression: Identifies the list of compression methods that the client supports.

ServerHello

If the server is able to find an acceptable set of algorithms, it responds to the ClientHello message with a ServerHello message. The server may use the ServerHello message to allow a resumed session. The ServerHello message contains some of the following components:

Version: The version field contains the highest SSL version supported by both the client and server.

Random: A random number generated by the server.

Session ID: Identifies a particular SSL session. If the client sends a non-zero session ID and the server locates a match in its cache, the server will attempt to respond with the same value as was supplied by the client, and resume the session using the same cipher suite.

Cipher suites: Identifies the cipher suite chosen by the server from the list of ciphers that the client supports.

Compression: Identifies the compression method chosen by the server from the list that the client supports.

Certificate

The server sends its Certificate message containing the server's certificate or list of (chain) certificates, depending on the selected cipher suite.

Note: The server may send a ServerKeyExchange message when the server Certificate message does not contain enough data to allow the client to exchange a premaster secret. This is true of some ciphers such as DHE-DSS.

ServerHelloDone

After sending its certificate, the server sends a ServerHelloDone message, indicating it is done with handshake negotiation.

ClientKeyExchange

The client sends the ClientKeyExchange message containing the PreMasterSecret. The PreMasterSecret is sent encrypted using the public key of the server.

ChangeCipherSpec

Both the client and server send the ChangeCipherSpec message after the security parameters have been determined. The ChangeCipherSpec message activates the negotiated SSL options for the session. From this point forward, all messages are authenticated and encrypted. This stage is significant as it indicates that subsequent records will be protected under the newly negotiated CipherSpec and keys.

Finished

Each party sends a Finished message under the new algorithm, keys and secrets. The Finished message indicates that the handshake is complete, and the parties may begin to exchange application layer data.

Identifying SSL handshake failures

When troubleshooting SSL handshake failures, it is important to identify the stage in which the failure occurs. For example, if the failure occurs during the initial negotiation phase, the client and server may not have agreed on the complete list of parameters, such as protocol version or cipher. For information about identifying handshake failures, refer to the following information:

Impact of procedure: Performing the following procedure should not have a negative impact on your system.

Negotiation stage

During the negotiation phase, the client starts the SSL communication between the two systems by presenting the SSL options to the server, and the server responds by selecting the options it supports. This stage defines the parameters for the secure channel. If the client and server do not agree on the complete list of options, the handshake will fail, often with very little diagnostic data. The most common failures during the negotiation stage involve the following incompatible components: protocols, ciphers, secure renegotiation options, or client certificate requests.

To understand failures in the negotiation stage, it is important to understand the client and server behavior during the message exchange.

  • The ClientHello offers the highest protocol version supported by the client. If the server does not support the client's protocol version, the server responds with a lower protocol version. The client then decides whether to downgrade the protocol or abort the SSL handshake.
  • The ClientHello also offers a list of supported cipher suites, in the preferred order. The server then typically chooses the highest cipher level shared by both. If the server does not support the ciphers from the client's list, the handshake will fail.

Negotiation phase handshake examples

  • Successful negotiation

    In the following example, the client offered protocol TLSv1.2 (version 3.3) and the server downgraded the protocol to TLSv1.0 (version 3.1). The server also chose the preferred cipher from the client's list:

    1 1 0.0003 (0.0003) C>SV3.3(79) Handshake
    ClientHello
    Version 3.3
    cipher suites
    TLS_RSA_WITH_RC4_128_SHA
    TLS_RSA_WITH_AES_128_CBC_SHA
    TLS_RSA_WITH_AES_256_CBC_SHA
    TLS_RSA_WITH_AES_128_CBC_SHA256
    TLS_RSA_WITH_AES_256_CBC_SHA256

    1 2 0.0008 (0.0005) S>CV3.1(74) Handshake
    ServerHello
    Version 3.1
    cipherSuite TLS_RSA_WITH_RC4_128_SHA

  • Unsuccessful negotiation

    In the following examples, the client and server fail to agree on the SSL protocol version in the first example, and the SSL cipher in the second example.

    Example 1: The client and server unsuccessfully negotiate the protocol. The server does not support protocol version below TLS1 (version 3.1) and the client does not support protocol versions above SSLv3 (version 3.0):

    1 1 0.0012 (0.0012) C>SV3.0(47) Handshake
    ClientHello
    Version 3.0
    cipher suites
    SSL_RSA_WITH_AES_256_CBC_SHA

    1 2 0.0013 (0.0000) S>CV0.0(2) Alert
    level fatal
    value handshake_failure

    Example 2: The client and server unsuccessfully negotiate a cipher; the server does not support any of the client's ciphers. This is a common failure:

    1 1 0.0012 (0.0012) C>SV3.1(58) Handshake
    ClientHello
    Version 3.2
    cipher suites
    TLS_DH_anon_WITH_RC4_128_MD5

    1 2 0.0013 (0.0000) S>CV3.2(2) Alert
    level fatal
    value handshake_failure

    Note: The SSL alert message (Alert 2 level fatal) is marginally useful and means an unrecoverable error has occurred. If the virtual server is using a Client SSL profile, you may be able to enable useful message logging by modifying the SSL logging level to debug.

ChangeCipherSpec (client)

During the client's ChangeCipherSpec phase, the client initializes the options that were negotiated by both parties. This phase marks the point when the parties change the secure channel parameters from using asymmetric (public key) to symmetric (shared key) encryption. A handshake failure during this phase may relate to SSL message corruption or issues with the SSL implementation itself.

ChangeCipherSpec (server)

During the server's ChangeCipherSpec phase, the server initializes the options that were negotiated by both parties. This phase marks the point when the parties change the secure channel parameters from using asymmetric (public key) to symmetric (shared key) encryption. A handshake failure during this phase may relate to SSL message corruption or issues with the SSL implementation itself.

Application phase

Messages marked as application_data indicate that data is being successfully encrypted. Failures in the application phase indicate application layer events. For example, a client's request for a document that results in an HTTP 500 error, may cause a failure during this phase. To diagnose failures during the application phase, you must decrypt the SSL session using a utility, such as ssldump.

Enabling SSL debug logging

You can enable SSL debug logging on the BIG-IP system, test SSL connections for the virtual server using a web browser or the OpenSSL client, and then review the debug log files. Doing so will provide more useful logging information when troubleshooting SSL handshake failures.

Note: Beginning in 12.0.0, the BIG-IP system automatically logs SSL handshake failure information through standard logging; the use of debug logging for SSL handshake failures is not required.

For example, with debug logging enabled, the system logs error messages similar to the /var/log/ltm file that appear similar to the following example:

  • The client and server unsuccessfully negotiate the protocol version:

    debug tmm3[9261]: 01260009:7: Connection error: ssl_hs_rxhello:4409: unsupported version (70)

  • The client and server unsuccessfully negotiate a cipher:

    debug tmm1[9261]: 01260009:7: Connection error: ssl_select_suite:4133: no shared ciphers (40)

To enable SSL debug logging, perform the following procedure:

Impact of procedure: F5 recommends that you return the SSL log level to the default value after you complete the troubleshooting steps. Leaving debug logging enabled when the system is in normal production mode may generate excessive logging and cause poor performance.

  1. Log in to the Traffic Management Shell (tmsh) by typing the following command:

    tmsh

  2. Enable SSL debug logging by typing the following command:

    modify /sys db log.ssl.level value Debug

Important: After you test SSL connections for the virtual server using a web browser or OpenSSL client, you should disable SSL debug logging by typing the following command:

modify /sys db log.ssl.level value Warning

Testing SSL connections (using s_client)

After you enable SSL debug logging on the BIG-IP system, you should test SSL connections for the virtual server using a web browser or other utility, such as the OpenSSL utility, s_client, or cURL. Using the s_client utility may provide additional debugging information that you can use to troubleshoot the issue. After making several requests to the virtual server, you can review and analyze the debug log files on the BIG-IP system.

To test SSL connections using the s_client, perform the following procedure:

Impact of procedure: Performing the following procedure should not have a negative impact on your system.

  1. Log in to the command line of a Linux host (with a current version of OpenSSL) that can access the SSL virtual server.
  2. Test SSL connections for the virtual server using the following command syntax:

    openssl s_client -connect <virtual_server>:<port>

    For example:

    openssl s_client -connect 10.12.23.115:443

  3. If the handshake attempt fails, take note of SSL errors returned by the s_client utility.
  4. If the handshake succeeds, type the following at the prompt:

    GET / HTTP/1.0

  5. Press Enter twice.

    The HTML page should display.

Reviewing log messages related to SSL handshake failures

After you test SSL connections using a web browser or OpenSSL client, you should review the BIG-IP log files for debug error messages related to the SSL handshake. To do so, perform the following procedure:

Impact of procedure: Performing the following procedure should not have a negative impact on your system.

  1. Log in to the BIG-IP command line.
  2. Use a Linux text utility, such as less or tail, to review the /var/log/ltm file.

    For example:

    tail -f /var/log/ltm

    Note: To filter the log information for SSL errors only, use the grep command.

    For example:

    cat /var/log/ltm |grep -i 'ssl'

  3. Review the debug logs for SSL handshake failure or SSL alert codes.

Packet tracing using the ssldump utility

The ssldump utility is a protocol analyzer for SSL/TLS that identifies TCP connections from a chosen packet trace or network interface and attempts to interpret the packets as SSL/TLS traffic. When the ssldump utility identifies SSL/TLS traffic, it decodes the records and displays them in text to standard output. If provided with the private key that was used to encrypt the connections, the ssldump utility may also be able to decrypt the connections and display the application data traffic. You can use the ssldump utility to examine, decrypt, and decode SSL-encrypted packet streams that are processed by the BIG-IP system. For information about using ssldump to troubleshoot SSL handshake failures, refer to SOL10209: Overview of packet tracing with the ssldump utility.

REF: SOL15292

  • 9 months later...

Generate CSR using a script

/usr/local/bin/csr

Usage:csr
csr -cname <Common_name> -refcode <Common_Name(or)Reference_code> -env <Environment>

-cname -> Name of the csr or key file name used to identification
-refcode is ref-code for private cert or common_name for public cert (16704968 or server.int.thezah.com)
-env: Dev, Model, Prod

Here is a break down of the script itself

#This script will generate CSRs with private key password encrypted.
#Different passwords used to encrypt key in different environments ( Dev/Model/Prod )
#It works well with refcodes as well as common names.

display_usage() { 
    echo -e "\nUsage:csr\ncsr -cname <Common_name> -refcode <Common_Name(or)Reference_code> -env <Environment>\n" 
    echo -e "-cname -> Name of the csr or key file name used to identification"
    echo -e "-refcode is ref-code for private cert or common_name for public cert (16704968 or server.int.thezah.com)"        
    echo -e "-env: Dev, Model, Prod"
} 
if [  $# -le 1 ] 
then 
        display_usage
        exit 1
fi 

dir="/apps/Certificates/"
cname=$2
refcode=$4
env=$6
cd $dir
while getopts -cname:-refcode:-env: opt; do
     case $opt in
         -cname)   cname="$OPTARG"      ;;
         -refcode) refcode="$OPTARG"    ;;
         -env)     env="$OPTARG"         ;;
         *)        echo "Invalid syntax";display_usage;exit 1;;
     esac
done


while true; do
    #read -p "Cert will be installed on which environment? (options:- M-Model / D-Dev / P-Prod)" env
    case $env in
         Dev) passwd=DeVPa55;break;;
         Model) passwd=mDLPa$$;break;;
         Prod) passwd=PRoDpa55;break;;
         *) echo "Please select a valid environment";display_usage; exit 1;;
    esac
done

if [ -f "$dir/${cname}.csr" ]
then 
    echo "CSR found and file name is $dir/${cname}.csr"
else
openssl req -new -sha256 -newkey rsa:2048 -nodes -out ${cname}.csr -keyout ${cname}.key.unencrypted -subj "/C=US/ST=NC/O='theZAh Corp'/CN=${refcode}"
openssl rsa -aes256 -in ${cname}.key.unencrypted -passout pass:$passwd -out ${cname}.key
rm ${cname}.key.unencrypted
echo "********************************************************************* "
cat ${cname}.csr
echo "********************************************************************* "
cat ${cname}.key
echo "********************************************************************* "
fi

 

 

What? You want to decrypt your SSL key?  Do you have the password?  If so, proceed below

openssl rsa -in <enc.key> -out decript.some.key

Enter password of encrypted key: 

 

To encrypt a private key using aes256

openssl rsa -aes256 -in someunencrypted.key -out nowencyrpted.key

 

Create an account or sign in to comment

Important Information

By using this site, you agree to our Terms of Use.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.