GeoIP with MaxMind

We don’t get ip geolocation out of the box with GoAccess. But we can integrate MaxMind’s geoIP database to get location support in our analytics.

Download the databases from MaxMind #

MaxMind publishes free databases each week, but you do need to register with them to be able to download. Create an account and add a license key. You should then be able to download the database files using your key:

# Database URL
https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=YOUR_LICENSE_KEY&suffix=tar.gz

# SHA256 URL
https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=YOUR_LICENSE_KEY&suffix=tar.gz.sha256

Let’s automate pulling these down with a script. Uncomment the LICENSE_KEY=“” line and add your license key, or set it from the command line when you run the script.

#!/bin/bash
set -e

cd "$HOME/src/maxmind"
TAR_FILE="GeoLite2-City.tar.gz"
SHA256_FILE="$TAR_FILE.sha256"
#LICENSE_KEY=""

DB_URL="https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=$LICENSE_KEY&suffix=tar.gz"
SHA256_URL="$DB_URL.sha256"

if [ -z $SKIP_DOWNLOAD ]; then
  curl -o "$SHA256_FILE" "$SHA256_URL"
  FRESH_FILE=$(awk '{print $2}' "$SHA256_FILE")
  curl -o "$FRESH_FILE" "$DB_URL"
else
  FRESH_FILE=$(awk '{print $2}' "$SHA256_FILE")
fi

sha256sum -c "$SHA256_FILE"

mv "$FRESH_FILE" "$TAR_FILE"
rm "$SHA256_FILE"

New files are released on Tuesday, so set this script to run on Wednesdays in crontab

crontab -e

1 0 * * 3 ~/src/maxmind/pull-geoip-db.sh

Add geoIP support in GoAccess report #

We now have a copy of the geoIP database that is updated weekly. We just have to tell GoAccess to use this db file now when we generate the reports. Add the following variables to the top of the script that generates the reports:

GEOIP_DB=“GeoLite2-City.mmdb”
LOCAL_GEOIP_DB=“$HOME/src/maxmind/$GEOIP_DB”
CONTAINER_GEOIP_DB=“/data/maxmind/$GEOIP_DB”

and then update the docker commands to mount the db and tell GoAccess to turn on geoIP support:

docker logs blog-prod 2> /dev/null \
  | awk ‘$1 ~ /[0-9]/ && $2 ~ /[0-9]/ && $3 ~ /10\.0\.0\.1/‘ \
  | docker run \
      --rm \
      -i \
      -e TZ=“America/New_York” \
      # mount the container
      -v "$LOCAL_GEOIP_DB":"$CONTAINER_GEOIP_DB" \
      allinurl/goaccess \
      -d \
      -o html \
      # turn on geoIP support
      --geoip-database "$CONTAINER_GEOIP_DB" \
      --log-format COMBINED \
      - > $DEPLOY_DIR/_site/goaccess-all-traffic.html
traffic.html

Putting it all together, the final script should look like this:

#!/bin/bash
if [ -z “$DEPLOY_DIR” ]; then
  echo “ERROR: DEPLOY_DIR not set”
  exit 1
fi

GEOIP_DB=“GeoLite2-City.mmdb”
LOCAL_GEOIP_DB=“$HOME/src/maxmind/$GEOIP_DB”
CONTAINER_GEOIP_DB=“/data/maxmind/$GEOIP_DB”

docker logs blog-prod *2*> /dev/null \
  | awk ‘$1 ~ /[0-9]/ && $2 ~ /[0-9]/ && $3 ~ /10\.0\.0\.1/‘ \
  | docker run \
      --rm \
      -i \
      -e TZ=“America/New_York” \
      -v "$LOCAL_GEOIP_DB":"$CONTAINER_GEOIP_DB" \
      allinurl/goaccess \
      -d \
      -o html \
      --geoip-database "$CONTAINER_GEOIP_DB" \
      --log-format COMBINED \
      - > $DEPLOY_DIR/_site/goaccess-all-traffic.html

docker logs blog-prod *2*> /dev/null \
  | awk ‘$1 ~ /[0-9]/ && $2 ~ /[0-9]/ && $3 ~ /10\.0\.0\.1/‘ \
  | grep -v “UptimeRobot” \
  | grep -v “Googlebot” \
  | docker run \
      --rm \
      -i \
      -e TZ=“America/New_York” \
      -v "$LOCAL_GEOIP_DB":"$CONTAINER_GEOIP_DB" \
      allinurl/goaccess \
      -d \
      -o html \
      --ignore-crawlers \
      --geoip-database "$CONTAINER_GEOIP_DB" \
      --log-format COMBINED \
      - > $DEPLOY_DIR/_site/goaccess-no-bots-traffic.html

docker logs blog-prod *2*> /dev/null \
  | awk ‘$1 ~ /[0-9]/ && $2 ~ /[0-9]/ && $3 ~ /10\.0\.0\.1/‘ \
  | grep “interwebs.gif” \
  | node ~/src/pixel-parser/index.js \
  | docker run \
      --rm \
      -i \
      -e TZ=“America/New_York” \
      -v "$LOCAL_GEOIP_DB":"$CONTAINER_GEOIP_DB" \
      allinurl/goaccess \
      -d \
      -o html \
      --ignore-crawlers \
      --geoip-database "$CONTAINER_GEOIP_DB" \
      --log-format COMBINED \
      - > $DEPLOY_DIR/_site/goaccess-tracked-pages.html

← Home