Troubleshooting DataSources in WildFly: The Complete Guide

1. Introduction

In Enterprise Java applications running on WildFly, database connectivity issues account for a large percentage of runtime failures. WildFly relies on the IronJacamar resource adapter architecture within its datasources subsystem to manage database connection pools. While adding a DataSource using defaults is simple, production environments quickly surface issues such as connection leaks, firewall drops, driver visibility errors, and pool exhaustion.

This guide provides a systematic, CLI-driven methodology for diagnosing, debugging, and resolving DataSource issues in WildFly.

2. Problem Statement

DataSource issues usually present themselves in application logs through one of four symptoms:

• Driver Registration Failures: The server cannot load the JDBC driver class or cannot find the registered module during server boot or deployment.
• Pool Exhaustion: Threads block indefinitely waiting for a connection until a timeout occurs (WFLYDS0022).
• Stale Connections: Firewalls or database servers terminate idle connections, causing application exceptions like Communication link failure or Connection reset when threads attempt to reuse pooled connections.
• Authentication/Network Failures: Incorrect JNDI names, invalid credentials, or network barriers block initial connection creation (JBAS010440).

3. Solution Overview

To resolve DataSource problems efficiently, follow this diagnostic workflow:

1. Module Verification: Ensure the JDBC driver is properly deployed as a JBoss Module with correct dependency definitions.
2. Pool Validation Configuration: Implement connection testing mechanisms (such as background-validation and valid-connection-checker-class-name) to flush broken connections automatically.
3. CLI Diagnostic Execution: Run runtime connection tests and read live pool metrics using WildFly's jboss-cli.sh tool.
4. Subsystem Trace Logging: Enable targeted log categories for org.jboss.as.connector and org.jboss.jca to expose underlying JCA events.

4. Step-by-Step Procedure

Step 1: Verify the JDBC Driver Configuration

The recommended approach for configuring JDBC drivers in WildFly is deploying them as custom modules under JBOSS_HOME/modules/. A missing or misconfigured module.xml file is the primary cause of driver load failures.

For example, a MySQL driver module placed in modules/system/layers/base/com/mysql/main/module.xml must look like this:

<module xmlns="urn:jboss:module:1.9" name="com.mysql">
    <resources>
        <resource-root path="mysql-connector-j-8.3.0.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>

Register the driver in the datasources subsystem via CLI:

/subsystem=datasources/jdbc-driver=mysql:add(driver-name=mysql, driver-module-name=com.mysql, driver-xa-datasource-class-name=com.mysql.cj.jdbc.MysqlXADataSource)

Step 2: Define the DataSource using CLI

Avoid editing standalone.xml manually while the server is running. Use standard management CLI commands to add and configure your DataSource:

/subsystem=datasources/data-source=MySQLDS:add( \
    jndi-name="java:jboss/datasources/MySQLDS", \
    driver-name="mysql", \
    connection-url="jdbc:mysql://localhost:3306/appdb?useSSL=false&serverTimezone=UTC", \
    user-name="dbuser", \
    password="dbpassword", \
    initial-pool-size=5, \
    min-pool-size=5, \
    max-pool-size=20, \
    blocking-timeout-wait-millis=5000 \
)

Step 3: Configure Validation to Prevent Stale Connections

Network firewalls often silently drop idle TCP connections. When an application requests a connection from the pool, it receives a dead socket, causing query execution to fail.

Enable robust validation settings directly via CLI:

/subsystem=datasources/data-source=MySQLDS:write-attribute(name=valid-connection-checker-class-name, value="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker")
/subsystem=datasources/data-source=MySQLDS:write-attribute(name=background-validation, value=true)
/subsystem=datasources/data-source=MySQLDS:write-attribute(name=background-validation-millis, value=30000)
/subsystem=datasources/data-source=MySQLDS:write-attribute(name=exception-sorter-class-name, value="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter")
/subsystem=datasources/data-source=MySQLDS:write-attribute(name=idle-timeout-minutes, value=5)

Step 4: Enable Detailed JCA & DataSource Logging

When connection creation fails without detailed error messages in server.log, increase the log level of the JCA resource adapter and datasources subsystems to DEBUG or TRACE using CLI:

/subsystem=logging/logger=org.jboss.as.connector:add(level=DEBUG)
/subsystem=logging/logger=org.jboss.jca:add(level=DEBUG)

To capture detailed connection pool allocation and leak details, raise the logging level to TRACE:

/subsystem=logging/logger=org.jboss.jca.core.connectionmanager:add(level=TRACE)

5. Verification

Testing Connections via CLI

Execute an immediate test against the pooled connections without triggering application logic:

/subsystem=datasources/data-source=MySQLDS:test-connection-in-pool

Expected successful output:

{
    "outcome" => "success",
    "result" => [true]
}

Monitoring Runtime Pool Statistics

WildFly disables runtime statistics by default to reduce overhead. Enable them to inspect connection usage:

/subsystem=datasources/data-source=MySQLDS:write-attribute(name=statistics-enabled, value=true)

Query pool metrics in real-time:

/subsystem=datasources/data-source=MySQLDS/statistics=pool:read-resource(include-runtime=true)

Key metrics to evaluate:

6. Troubleshooting Common Scenarios

Scenario 1: WFLYDS0022 - Did not receive connection within allocated timeout

Error Message:

javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/MySQLDS
...
Caused by: java.lang.Throwable: WFLYDS0022: Did not receive connection within allocated timeout of 5000 milliseconds

Root Cause: All connections in the pool (up to max-pool-size) are active and in use, and waiting threads exceeded blocking-timeout-wait-millis. This usually indicates a connection leak where application code opens connections without closing them in a finally block or try-with-resources statement.

Solution: Enable debug connection leaks via CLI to print the stack trace of threads holding unclosed connections:

/subsystem=datasources/data-source=MySQLDS:write-attribute(name=debug-unflushed, value=true)

Increase max-pool-size temporarily if demand legitimately exceeds capacity:

/subsystem=datasources/data-source=MySQLDS:write-attribute(name=max-pool-size, value=50)

Scenario 2: WFLYDS0015 - Is JDBC driver installed?

Error Message:

WFLYDS0015: Is JDBC driver installed? Cannot load driver class 'com.mysql.cj.jdbc.Driver'

Root Cause: The driver name listed in the DataSource definition does not match any registered driver in the datasources subsystem, or the driver module lacks the required JAR files.

Solution: List all registered drivers available to the server:

/subsystem=datasources:installed-drivers-list

Verify that the output matches the driver-name property assigned to your DataSource.

Scenario 3: Stale Connections after Database Restart or Firewall Idle Drops

Error Message:

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet successfully received from the server was 3,600,000 milliseconds ago.

Root Cause: The database server or an intermediate network firewall dropped idle TCP connections. WildFly's pool still considers these sockets valid.

Solution: Enable aggressive connection eviction and proactive background validation via CLI:

/subsystem=datasources/data-source=MySQLDS:write-attribute(name=validate-on-match, value=true)
/subsystem=datasources/data-source=MySQLDS:write-attribute(name=idle-timeout-minutes, value=3)

7. Conclusion

Troubleshooting WildFly DataSources relies on structured diagnostics: start by verifying your JDBC driver module, validate connections using built-in mechanisms, use CLI tools to run on-demand diagnostics, and enable detailed logging to isolate leaks. By combining proactive validation rules with active monitoring via WildFly's CLI, you can maintain reliable database connection management across production deployments.