Skip to content

Using a9s LogMe

Stream Application Logs to LogMe

To use a9s LogMe with an application, create a service instance and bind the service instance to your application. For more information on managing service instances, see Managing Service Instances with the cf CLI.

View the Service Offering

After the tile is installed, run cf marketplace to see logme service offering and its service plans.

1
2
3
4
5
$ cf marketplace
Getting services from marketplace in org test / space test as admin...
OK
service    plans                 description
logme      logme-xs, logme-m     This is the anynines LogMe service.

Create a Service Instance

Provision a LogMe instance with an arbitrary {instance_name}.

1
cf create-service {service_name} {plan_name} {instance_name}
1
cf create-service logme logme-xs my-logme-service

Depending on your infrastructure and service broker utilization, it may take several minutes to create the service instance.

Run the cf services command to view the creation status. This command displays a list of all your service instances. To view the status of a specific service instance, run cf service {instance_name}.

Bind the Logme Service to Your Application

After the LogMe service instance is created, bind the LogMe service to any applications whose logs should be streamed.

1
cf bind-service {app_name} {instance_name}
1
cf bind-service my-app my-logme-service

Restage or Restart Your Application

To enable your application to access the service instance, run cf restage or cf restart to restage or restart your application.

See Your Application's Logs

Perform the following steps to see your application's logs:

  1. Get the dashboard URL using cf service my-logme-service.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    Service instance: my-logme-service
    Service: logme
    Bound apps: my-app
    Tags:
    Plan: logme-xs
    Description: This is the anynines LogMe service.
    Documentation url: https://a9s-logme-dashboards.your-domain.com/service-instance/a89f3114-5e77-40a5-b3b0-34a9741f3cd5
    Dashboard:
    Last Operation
    Status: create succeeded
    Message:
    Started: 2017-07-22T17:10:21Z
    Updated: 2017-07-22T17:13:26Z
    
  2. Browse to the dashboard URL and authenticate on the redirected page with the "Login with WebKey" method:
    authentication-page

  3. Click Authorize to approve the authorization request:
    authorization-page
  4. In the dashboard that appears, specify the Index name or pattern and the Time-field name. For the Index name or pattern you can use the suggested value.
    dashboard-index For the Time-field name use the only available value, @timestamp.
    dashboard-timestamp
  5. Click Create to apply the settings.

Your application's logs appear on the Discover tab of the dashboard:
dashboard-app-logs

Stream a9s Service Logs to LogMe

To use a LogMe service instance to monitor another service instance, create an a9s LogMe service instance first.

Create a Service Key

After the LogMe service is instantiated, create a new key for your service instance.

1
cf create-service-key {instance_name} {service_key}
1
2
3
4
5
6
7
8
$ cf create-service-key my-logme-service key1
$ cf service-key my-logme-service key1

{
 "host": "syslog://d37f7da-logstash.service.dc1.consul:514",
 "password": "a9sfd6e0d814e78c903290ebb5a7207b20c5f0a2653",
 "username": "a9sed20b19c769f0804bc68b97d02cba86e9c3a0379"
}

Update Your Service

The cf update-service command used with the -c flag can let you stream your Syslog to a third-party service. In this case, the command expects a JSON string containing the syslog key. For this, you need to give the URL given by the cf service-key command as a value for the syslog key.

1
2
$ cf update-service service-instance-to-monitor \
-c '{"syslog": ["d37f7da-logstash.service.dc1.consul:514"]}'

See Your Service's Logs

Perform the following steps to see your service's logs:

  1. Get the dashboard URL with cf service my-logme-service.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    Service instance: my-logme-service
    Service: logme
    Bound apps: my-app
    Tags:
    Plan: logme-xs
    Description: This is the anynines LogMe service.
    Documentation url: https://a9s-logme-dashboards.your-domain.com/service-instance/a89f3114-5e77-40a5-b3b0-34a9741f3cd5
    Dashboard:
    Last Operation
    Status: create succeeded
    Message:
    Started: 2017-07-22T17:10:21Z
    Updated: 2017-07-22T17:13:26Z
    
  2. Browse to the dashboard URL and authenticate on the redirected page with your Cloud Foundry credentials:

    authentication-page 3. Click Authorize to approve the authorization request: authorization-page 4. The dashboard appears. Specify the Index name or pattern and the Time-field name. For the Index name or pattern you can use the suggested value. dashboard-index For the Time-field name use the only available value, @timestamp. dashboard-timestamp 5. Click Create to apply the settings.

Your service's logs appear on the Discover tab of the dashboard:

dashboard-service-logs

Stop Streaming Application Logs to LogMe

Follow the steps below to stop streaming the logs of your applications to LogMe.

List Available Services

Run cf services to list available service instances and get the name of the service instance you want to delete.

1
2
3
4
5
6
7
$ cf services

Getting services in org test / space test as admin...
OK

name                 service   plan         bound apps    last operation
my-logme-service     logme     logme-xs     logme-app     create succeeded

This example shows that the service instance my-logme-service is bound to the logme-app app.

Unbind the Service Instance

If your LogMe service instance is bound to an app, you need to unbind it first.

1
cf unbind-service {app_name} {instance_name}
1
cf unbind-service logme-app my-logme-service

Delete the Service Instance

1
cf delete-service {instance_name}
1
cf delete-service my-logme-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.

Custom Parameters

Data Retention

In order to clean up old Elasticsearch indices a tool called Curator runs periodically. By default, it deletes indices older than 30 days.

You can overwrite that configuration using the custom parameters curator_retention_unit and curator_retention_period, e.g.:

1
2
cf create-service a9s-logme logme-single-small my-logme-service -c '{ "curator_retention_unit": "days", "curator_retention_period": 90 }'
cf update-service my-logme-service -c '{ "curator_retention_unit": "hours", "curator_retention_period": 3 }'

For the curator_retention_unit you can use the following values: seconds, minutes, hours, days, weeks, months, years]. For the curator_retention_period you can use a positive integer value greater than zero.

Stop Streaming Service Logs to LogMe

Overwrite Your Service Configuration

Stop streaming your service instance logs to your LogMe instance, by updating the syslog key of your service configuration with an empty string.

1
cf update-service {instance_name} -c '{"syslog": [""]}'
1
cf update-service service-instance-to-monitor -c '{"syslog": [""]}'

Delete the Service Key and Instance

If you are not using it anymore, you may want to delete the service key and the service instance itself.

1
2
cf delete-service-key {instance_name} {service_key}
cf delete-service {instance_name}
1
2
cf delete-service-key my-logme-service key1
cf delete-service my-logme-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.

Handle Stack Traces

Java stack traces typically contain multiple lines of content, but Logstash and the Syslog protocol do not support multiline log events out-of-the-box. Instead, they store each line as an individual event in Elasticsearch. Prevent this by encoding the newline character \n as \u2028 (line separator). This enables Syslog and Logstash to read the stack trace as one log event, but is converted to newline when saving it to Elasticsearch.

Adjust the logging configuration of your application to use the encoded newline character \u2028 instead of the default \n line separator.

Examples for Java Spring Configuration

This section shows how to configure the logging settings for multiline log events using the application properties or YAML/XML files.

Add the following snippet to the application's logging configuration:

1
%replace(%xException){'\n','\u2028'}%nopex

This snippet converts every occurrence of \n in the stack trace to \u2028 (line separator) and filters the original text from the log using the %nopex pattern.

Example Configuration in application.properties

In this example the logging.pattern.console property in the application.properties file is set to log the date, time and log message followed by a newline character as shown below.
Encode linebreaks inside exception messages by adding the given snippet in front of the newline character. This applies the encoding to stack traces of exceptions, but leaves other log events unmodified.

1
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss} - %msg%n
1
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss} - %msg%replace(%xException){'\n','\u2028'}%nopex%n

Example Configuration in logback-spring.xml

In this example the pattern property in the STDOUT section is set to log the date, time and log message followed by a newline character as shown below.
Encode line breaks inside exception messages by adding the given snippet in front of the newline character. This applies the encoding to stack traces of exceptions, but leaves other log events unmodified.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss} - %msg%n</pattern>
    </encoder>
  </appender>
  <root level="debug">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss} - %msg%replace(%xException){'\n','\u2028'}%nopex%n</pattern>
    </encoder>
  </appender>
  <root level="debug">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

In contrast to the application.properties, XML/YAML configuration is defined per class. Therefore this snippet must be added for every class that logs exceptions to STDOUT. Any logging configuration that supports the format %replace(%xException){'\n','\u2028'}%nopex can be configured accordingly.

Any questions left?

Ask the community


Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.