how to consume python logs inside a container with Datadog

This weekend I was adding finishing touches to photoRake. Deploying it to railway has been delightful. They're cheaper, faster, and easier to use than the alternatives. You give them a docker image, and they handle the rest.

That said, setting up Datadog for log ingestion was neither obvious nor trivial. Datadog suggests running their agent in a sidecar container, which is sadly not an option for railway apps as of yet. I needed to run my agent inside the container.

I opted to simply install the agent within my docker container. It bloats your container a little bit, but I'd rather do that than manage my own container cluster.

Here's how I got Datadog log ingestion working for photoRake:

1. in your Dockerfile, add the following to install the agent:

RUN DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=<API_KEY> DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)" 

This will install the agent within a linux based docker image (my base image for photoRake is python:3.9, which runs on linux).

2. Configure the agent with a datadog.yaml file. My file was copied from the official Datadog repo, with a single modification (line 3) to enable log ingestion .

api_key: <API_KEY>
site: datadoghq.com
logs_enabled: true

3. Make a conf.yaml to help the agent find your application logs. Since photoRake is a python app, I followed these conf.yaml instructions.

init_config:

instances:

logs:

  - type: file
    path: <path-to-your-logs-within-container>
    service: <service-name>
    source: python
    sourcecategory: sourcecode

3. Copy your configs to their respective locations in your Dockerfile

COPY datadog.yaml /etc/datadog-agent/datadog.yaml
COPY conf.yaml /etc/datadog-agent/conf.d/python.d/conf.yaml

4. And finally, make sure you chown the copied config files, so the datadog agent can use them

RUN chown dd-agent:dd-agent \
  /etc/datadog-agent/datadog.yaml \
  /etc/datadog-agent/conf.d/python.d/conf.yaml

summarize, here is my Dockerfile with the above changes:

FROM python:3.9

...

# Datadog setup
RUN DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=<API_KEY> DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"

COPY conf.yaml /etc/datadog-agent/conf.d/python.d/conf.yaml

COPY datadog.yaml /etc/datadog-agent/datadog.yaml

RUN chown dd-agent:dd-agent /etc/datadog-agent/datadog.yaml /etc/datadog-agent/conf.d/python.d/conf.yaml

...

And that's it - you've set up a Datadog agent for log consumption within a container.