This tutorial discusses how to validate Datasource connections using in WildFly using proper validation methods. Besides, it will discuss how to troubleshoot database connections which are not properly replaced when Database connections are no longer valid. This is even more important today than it used to be: in containerized and cloud-native deployments, transient connection loss caused by pod rescheduling, service mesh sidecars or managed database failovers is the norm rather than the exception, so getting validation right directly affects how resilient your application is.

Connection Validation in a nutshell

A datasource connection validation helps to ensure that the connections to the database are still valid. The strategy behind connection validation is to periodically test connections from the connection pool to see if they are still usable. If a connection fails validation, it is removed from the pool and a new one is created to replace it.

This feature is useful for several reasons:

• It helps detect and prevent stale connections, which can occur after a database restart. It can also happens because of a network connection outage, causing the connections in the pool to become invalid.
• It can also help to detect and prevent potential security issues, as it can detect unauthorized access to the database, lost credentials or malicious access.

For example, consider the Closed Connection error:

Caused by: java.sql.SQLRecoverableException: Closed Connection
        at oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:4051)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3563)
        at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)
        at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1493)
        at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:111)

This error can happen if there’s something wrong with your firewall / routing settings, and the connections are closed/dropped somewhere in the middle of the route.

To solve this issue, you can manage database connection validation using the validation element within the datasource section of the configuration file. The validation of a connection implies the following steps:

1) Choose a validation method

Firstly, choose between validate-on-match and background-validation.

validate-on-match

You can apply the validate-on-match option on your Datasource as follows:

/subsystem=datasources/data-source=ExampleDS:write-attribute(name=validate-on-match,value=true)

Then, the ExampleDS datasource now includes the following element:

<validate-on-match>true</validate-on-match>

With validate-on-match, the database connection is validated every time it is checked out from the connection pool using the validation mechanism specified in the next step. If a connection is invalid, you will see a warning in the log and it retrieves the next connection in the pool. This continues until a valid connection is found. If you prefer not to cycle through every connection in the pool, you can use the use-fast-fail option.

Finally, if no valid connection is found a new connection is created. If the connection creation fails, an exception is returned to the requesting application.

Please notice that this setting provides the fastest recovery but creates the highest load on the database, therefore it might not be ideal for performance reasons.

background-validation

You can apply the background validation setting as follows:

/subsystem=datasources/data-source=ExampleDS:write-attribute(name=background-validation,value=true)

Which produces the following configuration:

<background-validation>true</background-validation>

The background-validation parameter, however, has been deprecated. It is sufficient to specify the background-validation-millis time in your Datasource configuration to activate it:

<background-validation-millis>10000</background-validation-millis>

It means that a background validation will take place every ms as specified by background-validation-millis. The default value for the background-validation-millis parameter is 0 milliseconds, meaning it is disabled by default. You should not set this attribute to the same value as your idle-timeout-minutes setting.

The lower the value of background-validation-millis, the more frequently the pool is validated and the sooner invalid connections are removed from the pool. However, lower values take more database resources. Higher values result in less frequent connection validation checks and use less database resources. However dead connections could remain in the pool for longer periods of time.

Please note that if the <validate-on-match> option is set to true, the background validation should be disabled. Also the reverse is true.

Finally note that WildFly only validates free connections during each background validation pass. Behind the hoods, each connection has a timestamp for when it was last checked. During a given background validation pass, the “last checked” timestamp is compared to the background-validation-millis value.

Here is a sample datasource which uses background-validation:

<datasource jta="true" jndi-name="java:jboss/datasources/mydb" pool-name="DemoDB" enabled="true" use-ccm="false">
      <connection-url>jdbc:mysql://localhost:3306/demodb?autoReconnect=true&amp;useUnicode=true&amp;characterEncoding=UTF-8</connection-url>
      <driver-class>com.mysql.jdbc.Driver</driver-class>
      <driver>mysql-connector-java-5.1.31-bin.jar</driver>
      <security>
          <user-name>user</user-name>
          <password>password</password>
      </security>
      <validation>
          <check-valid-connection-sql>select 1</check-valid-connection-sql>
          <validate-on-match>false</validate-on-match>
          <background-validation-millis>10000</background-validation-millis>
      </validation>
      <statement>
          <share-prepared-statements>false</share-prepared-statements>
      </statement>
    </datasource>

Note: this example keeps the historical mysql-connector-java-5.1.31-bin.jar module name for continuity with older setups you may still encounter. On a modern install, remember that Oracle renamed the artifact starting with the 8.x line to mysql-connector-j — this only affects the driver module, not the validation settings shown above.

2) Choose a validation mechanism

You can either specify a <valid-connection-checker> Class Name or a <check-valid-connection-sql>. Let’s see both options when using Oracle as Database.

valid-connection-checker

Here is a sample definition:

<validation>
        <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker"/>
        <validate-on-match>true</validate-on-match>
 
 </validation>

The JDBC Driver includes out of the box a Connection Checker Class to check if the Connection is Valid. Besides, you can also use a custom Connection Checker by packaging it in a module. Then, you can reference the Custom Connection Checker Class and the module in the data-source CLI as follows:

data-source add --name=MySqlDS --jndi-name=java:/MySqlDS --driver-name=mysql --connection-url=jdbc:mysql://localhost:3306/mydb --user-name=admin --password=admin --validate-on-match=true --background-validation=false --valid-connection-checker-class-name=com.sample.MySQLValidConnectionChecker --valid-connection-checker-module=my.module --exception-sorter-class-name=com.sample.MySQLExceptionSorter --exception-sorter-module=my.module

check-valid-connection-sql

Here is a sample definition:

<validation>
        <check-valid-connection-sql>SELECT 1 FROM DUAL</check-valid-connection-sql> 
        <validate-on-match>true</validate-on-match>
 
 </validation>

The recommended option is to specify a <valid-connection-checker> Class Name for your specific Database vendor.

🔍 Troubleshooting Validation Issues in WildFly

If you’re using WildFly and notice that dead or stale database connections are not being discarded or revalidated properly, you’re likely facing a connection validation issue in the datasource pool. Here’s a trace snippet showing such a situation:

Caused by: java.sql.SQLNonTransientConnectionException: (conn=210181) Connection is closed

And a diagnostic log:

Returning connection to pool ... lastValidated=1748083905749 ... connection handles=0 ...

Despite the connection being stale or closed, it appears that WildFly returned it to the pool as if it were still valid.

🧠 Root Cause: Missing exception-sorter

The exception-sorter is a critical component in datasource configuration. It tells WildFly how to interpret SQL exceptions — specifically, which ones indicate that a connection is no longer usable.

Without it, WildFly (IronJacamar under the hood) may:

• Not detect that a connection has failed (e.g. due to a database restart, timeout, or network issue)
• Return the connection to the pool as if it were healthy
• Re-use a dead connection, resulting in errors like Connection is closed

Fix: Add the appropriate exception-sorter for your database. For example, for MariaDB/MySQL:

/subsystem=datasources/data-source=AppDS:write-attribute(name=exception-sorter-class-name, value=org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter)

✅ Other Possible Causes & Fixes

Validation query returns incorrectly

The validation SQL (check-valid-connection-sql) must be valid for the DB and user privileges. For example:

• SELECT 1 → good for most databases
• SELECT current_date → fails if the user lacks privileges

Try executing the query manually from the same DB user.

Timeouts or server restarts

If the DB server disconnects idle connections (e.g. wait_timeout in MySQL), and background validation isn’t frequent enough, the pool might hold stale connections.

Fix:

• Reduce background-validation-millis to e.g. 30000 (30 seconds)
• Enable use-fast-fail to quickly dispose of bad connections

How to let JBoss / WildFly automatically reconnect to the DB using validation

Let’s see now a proof of concept example on using validation parameters to reconnect to the DB after a failure. Our initial configuration does not use validation of DB Connections:

<datasource jndi-name="java:/PostGreDS" pool-name="PostgrePool">
    <connection-url>jdbc:postgresql://localhost:5432/postgres</connection-url>
    <driver>postgres</driver>
    <security>
        <user-name>postgres</user-name>
        <password>postgres</password>
    </security>
</datasource>

Within our application, we are using the PostgreSQL Datasource:

@Resource(lookup="java:/PostGreDS")
   private DataSource ds;

Note: on WildFly 27 and newer, running on Jakarta EE 10, this annotation is imported from jakarta.annotation.Resource rather than the legacy javax.annotation.Resource package. The datasource validation configuration below is unaffected by this change.

Let’s stop and start the DB and see what happens. If we try accessing the Datasource here is what happens:

12:43:19,596 ERROR [stderr] (default task-1) org.postgresql.util.PSQLException: This connection has been closed.
12:43:19,597 ERROR [stderr] (default task-1) 	at org.postgres@42.2.5//org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:783)
12:43:19,602 ERROR [stderr] (default task-1) 	at org.postgres@42.2.5//org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:1680)

So, you have attempted to use a Connection which is closed. You can still remedy without an application restart, by flushing connections in the Pool:

/subsystem=datasources/data-source=PostgrePool:flush-all-connection-in-pool

However, to discard stale connection automatically, we should add validation checks to our datasource:

<validation>
    <validate-on-match>true</validate-on-match>
    <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLValidConnectionChecker"/>
</validation>

When validation checks are in place and a database restart happens, the outage won’t impact your applications. There is however a WARN in your logs which indicates the connection replacement, upon failure in validation:

12:49:57,046 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (default task-1) IJ000621: Destroying connection that could not be validated: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@73a8363f[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@2349e1b3 connection handles=0 lastReturned=1612698575441 lastValidated=1612698575417 lastCheckedOut=1612698575417 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@4b6aaa9f mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@4086c17e[pool=PostgrePool] xaResource=LocalXAResourceImpl@256ef25[connectionListener=73a8363f connectionManager=761b50a5 warned=false currentXid=null productName=PostgreSQL productVersion=10.5 (Debian 10.5-2.pgdg90+1) jndiName=java:/PostGreDS] txSync=null]

Connection Validation in Cloud-Native and Kubernetes Deployments

The validation strategies discussed so far become even more relevant once WildFly runs as a container in Kubernetes or OpenShift. In these environments, a “closed connection” error is rarely caused by a full database outage — it is usually the side effect of infrastructure churn that is completely normal in a cloud-native setup:

• Service mesh sidecar restarts: if your pod runs behind an Istio or Linkerd sidecar proxy, a sidecar restart (for a certificate rotation, a mesh upgrade, or a simple rescheduling) silently drops any TCP connection routed through it, even though both WildFly and the database are perfectly healthy.
• Managed database failovers: cloud-managed databases such as Amazon RDS Multi-AZ, Aurora, or Azure SQL failover groups intentionally terminate existing connections during a failover event, which can take anywhere from a few seconds to over a minute.
• Node eviction and pod rescheduling: when Kubernetes evicts or reschedules a database pod (for a self-managed DB running in-cluster), any existing connection becomes instantly stale, with no TCP-level notice sent to the client.

In all three cases, the fix relies on the same building blocks already described above — validate-on-match combined with use-fast-fail and a correctly configured exception-sorter — but the tuning should change: expect these events to happen far more often than in a traditional on-premise deployment, so err on the side of a lower background-validation-millis value and make sure use-fast-fail is enabled.

Tying validation into MicroProfile Health readiness probes

WildFly exposes health data through Eclipse MicroProfile Health, which Kubernetes uses to drive liveness and readiness probes. By default, the built-in readiness checks only confirm that the server itself is up — they say nothing about the datasource. You can close that gap with a custom @Readiness health check that borrows a connection from the pool, letting the validate-on-match logic you already configured decide whether the pool is actually usable:

@Readiness
@ApplicationScoped
public class DatasourceReadinessCheck implements HealthCheck {

    @Resource(lookup = "java:/PostGreDS")
    private DataSource ds;

    @Override
    public HealthCheckResponse call() {
        try (Connection c = ds.getConnection()) {
            return HealthCheckResponse.up("datasource-connection");
        } catch (SQLException e) {
            return HealthCheckResponse.down("datasource-connection");
        }
    }
}

This check is deliberately simple: it just borrows a connection and lets the datasource’s own validation mechanism weed out a stale one before it ever reaches the try-block. If the pool cannot produce a valid connection, the pod is reported as not ready, and Kubernetes stops routing traffic to it until validation restores a healthy connection — instead of letting requests fail against a pod that looks alive but can’t reach its database.

TLS/SSL Certificate Rotation: Another Cause of “Closed” Connections

A less obvious, increasingly common cause of the same symptoms described in the troubleshooting section above is certificate rotation. If your datasource connects over TLS — now the default expectation for most managed cloud databases — and the certificate used by the database, or a short-lived client certificate issued by a tool like HashiCorp Vault or cert-manager, is rotated while connections are checked out from the pool, you can see errors that look identical to a plain network drop but are actually an SSLException or handshake failure wrapped inside a generic SQLException.

Two things are worth checking if you suspect this is happening:

• Confirm that your exception-sorter actually classifies TLS/SSL-related exceptions as fatal. The vendor-provided exception sorters (MySQL, PostgreSQL, Oracle, etc.) are generally written with plain connectivity failures in mind and may not treat every SSL handshake failure the same way as a “connection closed” error — if in doubt, test it by forcing a certificate rotation in a non-production environment and checking whether the pool discards the affected connections.
• If certificates are rotated on a fixed schedule (for example, every 24 or 72 hours), align background-validation-millis so that a validation pass comfortably falls within that window, rather than relying purely on validate-on-match to catch the failure on next use.

This scenario is easy to misdiagnose as a database or network problem, when the actual fix is entirely on the validation/exception-handling side of the datasource configuration.

Conclusion

This article was a complete walk through datasource validation in JBoss / WildFly, from the classic validate-on-match / background-validation trade-off, through common misconfiguration issues, up to the failure modes you’re more likely to run into today: service mesh sidecar restarts, managed database failovers, and TLS certificate rotation in Kubernetes environments. Getting validation right lets your application recover from all of these automatically, instead of requiring a redeploy or a manual pool flush. If instead you’re chasing connections that leak out of the pool entirely and never get returned — rather than going stale — that’s a related but distinct problem, covered separately.