{"id":1290,"date":"2016-08-23T21:15:13","date_gmt":"2016-08-24T04:15:13","guid":{"rendered":"https:\/\/galencharlton.com\/blog\/?p=1290"},"modified":"2016-08-23T21:15:13","modified_gmt":"2016-08-24T04:15:13","slug":"visualizing-the-global-distribution-of-koha-installations-from-debian-packages","status":"publish","type":"post","link":"https:\/\/galencharlton.com\/blog\/2016\/08\/visualizing-the-global-distribution-of-koha-installations-from-debian-packages\/","title":{"rendered":"Visualizing the global distribution of Koha installations from Debian packages"},"content":{"rendered":"

A picture is worth\u00a0a thousand words:<\/p>\n

\"Downloads<\/a>
Click to get larger image.<\/figcaption><\/figure>\n

This represents the approximate geographic distribution of downloads of the Koha Debian packages over the past year. Data was taken from the Apache logs from debian.koha-community.org, which MPOW hosts. I counted only completed downloads of the koha-common package, of which there were over 25,000.<\/p>\n

Making the map turned out to be an opportunity for me to learn some Python. I first adapted a Python script I found<\/a> on Stack Overflow to query freegeoip.net<\/a>\u00a0and get the latitude and longitude\u00a0corresponding to each of the 9,432 distinct IP addresses that had downloaded the package.<\/p>\n

I then fed the results to OpenHeatMap<\/a>. While that service is easy to use and is written\u00a0with GPL3 code<\/a>, I\u00a0didn’t quite like the fact that the result is delivered via\u00a0an Adobe Flash embed. \u00a0Consequently, I turned my attention to Plotly<\/a>, and after some work, was able to write a Python script that does\u00a0the following:<\/p>\n

    \n
  1. Fetch the CSV file containing the coordinates and number of downloads.<\/li>\n
  2. Exclude as outliers rows where a given IP address made more than 100 downloads of the package during the past year \u2014 there were seven of these.<\/li>\n
  3. Truncate the latitude and longitude to one decimal place \u2014 we need not pester\u00a0corn farmers in Kansas for bugfixes<\/a>.<\/li>\n
  4. Submit\u00a0the dataset to Plotly with which to generate a\u00a0bubble map.<\/li>\n<\/ol>\n

    Here’s the code:<\/p>\n

    #!\/usr\/bin\/python\r\n\r\n# adapted from example found at https:\/\/plot.ly\/python\/bubble-maps\/\r\n\r\nimport plotly.plotly as py\r\nimport pandas as pd\r\n\r\ndf = pd.read_csv('http:\/\/example.org\/koha-with-loc.csv')\r\ndf.head()\r\n\r\n# scale factor the size of the buble\r\nscale = 3\r\n\r\n# filter out rows where an IP address did more than\r\n# one hundred downloads\r\ndf = df[df['value'] <= 100]\r\n\r\n# truncate latitude and longitude to one decimal\r\n# place\r\ndf['lat'] = df['lat'].map('{0:.1f}'.format)\r\ndf['lon'] = df['lon'].map('{0:.1f}'.format)\r\n\r\n# sum up the 'value' column as 'total_downloads'\r\naggregation = {\r\n    'value' : {\r\n        'total_downloads' : 'sum'\r\n    }\r\n}\r\n\r\n# create a DataFrame grouping by the truncated coordinates\r\ndf_sub = df.groupby(['lat', 'lon']).agg(aggregation).reset_index()\r\n\r\n\r\ncoords = []\r\npt = dict(\r\n    type = 'scattergeo',\r\n    lon = df_sub['lon'],\r\n    lat = df_sub['lat'],\r\n    text = 'Downloads: ' + df_sub['value']['total_downloads'],\r\n    marker = dict(\r\n        size = df_sub['value']['total_downloads'] * scale,\r\n        color = 'rgb(91,173,63)', # Koha green\r\n        line = dict(width=0.5, color='rgb(40,40,40)'),\r\n        sizemode = 'area'\r\n    ),\r\n    name = '')\r\ncoords.append(pt)\r\n\r\nlayout = dict(\r\n        title = 'Koha Debian package downloads',\r\n        showlegend = True,\r\n        geo = dict(\r\n            scope='world',\r\n            projection=dict( type='eckert4' ),\r\n            showland = True,\r\n            landcolor = 'rgb(217, 217, 217)',\r\n            subunitwidth=1,\r\n            countrywidth=1,\r\n            subunitcolor=\"rgb(255, 255, 255)\",\r\n            countrycolor=\"rgb(255, 255, 255)\"\r\n        ),\r\n    )\r\n\r\nfig = dict( data=coords, layout=layout )\r\npy.iplot( fig, validate=False, filename='koha-debian-downloads' )<\/pre>\n
    An interactive version of the bubble map is also available<\/a>\u00a0on Plotly.<\/p>\n
    Share this:<\/h3>