How to Enable Structured Logging
Good news! There is nothing to do for a brand new install. The log4j-default.xml file that comes with Kinetic Request CE located in the WEB-INF/config directory is already configured to start writing to the structured log files.
Upgrading Kinetic Request CE
If you have customized the log4j.xml or log4j-default.xml file, you'll need to append the following text in your log4j.xml file if you wish to enable the new structured logs. These lines can be added anywhere inside the >log4j:configuration ...< XML element in your customized log4j xml file, but we suggest adding it before the >!-- LOGGERS --< line. For example of an example of where to place the additional lines, take a look at the new WEB-INF/config/log4j-default.xml file that comes with the 2.0.2 Kinetic Request CE war file.
<!-- Structured Access File Appender -->
<appender name="structured-access-file-appender" class="org.apache.log4j.RollingFileAppender">
<param name="Append" value="true"/>
<param name="File" value="${com.kineticdata.core.dataDirectory}/logs/structured.access.log"/>
<param name="MaxFileSize" value="20MB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %X{app.instanceId} %X{app.sessionId} %X{app.correlationId} %X{app.requestId} %X{app.space} %X{app.user} %X{access.addressChain} - %X{access.requestMethod} %X{access.requestPath}%X{access.requestQuery} %X{access.responseStatus} %X{access.responseTime}ms%n"/>
</layout>
</appender>
<!-- Structured Authentication File Appender -->
<appender name="structured-authentication-file-appender" class="org.apache.log4j.RollingFileAppender">
<param name="Append" value="true"/>
<param name="File" value="${com.kineticdata.core.dataDirectory}/logs/structured.authentication.log"/>
<param name="MaxFileSize" value="20MB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %X{app.instanceId} %X{app.sessionId} %X{app.correlationId} %X{app.requestId} %X{app.space} %X{app.user} %X{access.addressChain} - %X{auth.event} %X{auth.strategy} PRINCIPAL:%X{auth.principal} %m%n"/>
</layout>
</appender>
<!-- Structured Application File Appender -->
<appender name="structured-application-file-appender" class="org.apache.log4j.RollingFileAppender">
<param name="Append" value="true"/>
<param name="File" value="${com.kineticdata.core.dataDirectory}/logs/structured.application.log"/>
<param name="MaxFileSize" value="20MB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %X{app.instanceId} %X{app.sessionId} %X{app.correlationId} %X{app.requestId} %X{app.space} %X{app.user} %X{access.addressChain} %5p [%t] - %m%n"/>
</layout>
<filter class="com.kineticdata.commons.logging.ContextFilter">
<param name="KeyToMatch" value="app.requestId" />
<param name="PatternToMatch" value=".+" />
</filter>
</appender>
<!-- Structured System File Appender -->
<appender name="structured-system-file-appender" class="org.apache.log4j.RollingFileAppender">
<param name="Append" value="true"/>
<param name="File" value="${com.kineticdata.core.dataDirectory}/logs/structured.system.log"/>
<param name="MaxFileSize" value="20MB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %X{app.instanceId} %5p [%t] - %m%n"/>
</layout>
<filter class="com.kineticdata.commons.logging.ContextFilter">
<param name="KeyToMatch" value="app.requestId" />
<param name="PatternToMatch" value="^$" />
</filter>
</appender>
Finally, restart Tomcat or the Kinetic Request CE web application.
CE Structured Logging Variables
Overview
The structured logging variables are variables that are available to use to format a log message in a format you want. The sub-section 'Upgrading Kinetic Request CE' above shows a sample of how these variables, which are highlighted, are used in the log4j-default.xml and log4j.xml files to configure the output format in the structured.access.log, structured.application.log, structured.auth.log, and structured.system.log files.
Variable Definitions
Variable | Definition | |
---|---|---|
access.addressChain | Either the IP addresses listed in the X-Forwarded-For HTTP header or the IP address establishing the request to Kinetic Core if no X-Forwarded-For HTTP header is present. Examples for the IP address establishing the request include a load balancer's IP address, or the client's IP address if no load balancer or reverse proxy is in front of Kinetic Request. | |
access.requestMethod | The HTTP request method used for the access request | |
access.requestPath | Portion of the request uri after the host and before the query string, ie `/kinetic/app/api/v1/version | ` |
access.request | Protocol Protocol used for the request - Example: HTTP/1.1 | |
access.request | Query Provides the request query string, starting with “?”. The request query string is the part of a URL proceeding the request path and the start of the request query is indicated by the question mark character. | |
access.response | Status The HTTP response code returned from Kinetic Request. | |
access.response | Time The amount of time taken to process the request on the server side in milliseconds. This does not include any network latency. | |
app.correlationId | This is an id that is generated for each end user HTTP request, passed between kinetic components to associate internal calls | |
app.instanceId | A consistent key based upon the server’s network interfaces and webapp location, can be used when aggregating logging to determine the source of the log message | |
app.requestId | A unique ID generated for each received HTTP request | |
app.sessionId | A user's session id or NO_SESSION for sessionless interaction; such as HTTP Basic API calls | |
app.space | The space slug for which the logged event occurred within | |
app.user | The username of the user associated to the logged event | |
auth.event | This is the type of the authentication event that occurred. Valid options are LOGIN, LOGIN_FAILURE, LOGOUT, CHECK, EXPIRE. |
Because the application always allows local authentication (default CE form based authentication) as a fallback, there will only be LOGIN_FAILURE messages when all of the configured security strategies fail AND local authentication fails (this means pre-authentication strategies like x509 will never have LOGIN_FAILURE messages). CHECK represents an unsuccessful authentication check for a security strategy which indicates the next security strategy will be tried
auth.principal The authentication principal is the unique identifier from an authentication strategy for a user, before they're mapped to a local CE username.
auth.strategy The authentication strategy used for this logged event. Some example strategies are: ldap, ldap/basic, local, local/basic, x509, saml
Updated over 1 year ago