#+OPTIONS:   H:3 num:nil toc:nil \n:nil @:t ::t |:t ^:nil -:t f:t *:t TeX:t LaTeX:t skip:t d:t tags:not-in-toc creator:t timestamp:nil author:nil title:nil
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="../../../css/stylesheet-blog.css">
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="../../../css/theorems-en.css">
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="../../../fontawesome/css/all.css" />
#+HTML_HEAD: <link rel="stylesheet" href="../../../highlight/styles/base16/solarized-light.min.css">
#+HTML_HEAD: <script src="../../../highlight/highlight.min.js"></script>
#+HTML_HEAD: <script>hljs.highlightAll();</script>
#+LANGUAGE: en-GB
#+STARTUP: latexpreview
#+TITLE: Plotting a GDP time series
#+DATE: September 2022
#+AUTHOR: Stéphane Adjemian
#+EMAIL: stephane.adjemian@univ-lemans.fr
#+PROPERTY: header-args:python :python /tmp/blog-plt-ts/bin/python

To plot a GDP time series (as in the slides of the introductory chapter of the
course, [[https://le-mans.adjemian.eu/croissance/cours/introduction.pdf][here]], in French) one first has to find and download some data. As in
the course, I use the [[https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2020][Maddison Project Database]], which provides (very) long
run data on GDP and population for more than 160 countries[fn:1: Data over
very long periods, more than a century, are only available for a limited
number of countries]. To download the data, which come as an Excel file, I use
a small Python script:

#+BEGIN_SRC bash :results silent :exports none :async t
  python3 -m venv /tmp/blog-plt-ts
  source /tmp/blog-plt-ts/bin/activate
  pip install pandas openpyxl matplotlib
#+END_SRC


#+begin_src python :session plt-ts-en :exports code :tangle gdppc-en.py :results none
  import urllib.request as url
  MADDISON = 'mpd2020'
  MADDISON_PATH = 'https://www.rug.nl/ggdc/historicaldevelopment/maddison/data/'+MADDISON+'.xlsx'
  File = url.urlopen(MADDISON_PATH)
  data = File.read()
  with open('./'+MADDISON+'.xlsx', 'wb') as f:
      f.write(data)
#+end_src

This script downloads an Excel file from the [[https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2020][Maddison Project Database]] website
and saves it locally on my disk. Since I do not like Excel files, I convert the
file =mpd2020.xlsx= into a =csv= (Comma Separated Values) text file:

#+begin_src python :session plt-ts-en :exports code :tangle gdppc-en.py :results none
  import pandas as pd
  data = pd.read_excel(MADDISON+'.xlsx', 'Full data', dtype=str, index_col=None)
  data.to_csv(MADDISON+'.csv', encoding='utf-8', index=False)
#+end_src

The file =mpd2020.csv= is saved to disk. It is a text file (which you can
therefore read with any text editor, or process programmatically, for instance
to build charts). The file contains a table. The first column gives a
three-letter country code, the second the name of the country, the third the
year, the fourth GDP per capita (in 2011 dollars) and the fifth the population
in thousands. For instance, rows 2 to 73 contain observations for Afghanistan
from 1820 to 2018. Note that the years are not necessarily consecutive (it
depends on the country and the period) and that more or less data are
available depending on the country (it depends on the history and culture of
the country, more or less inclined towards statistics).

In the course, I produced the charts with [[https://www.mathworks.com/products/matlab.html][Matlab]], a scientific computing
language (the codes used for the charts of the course are available [[https://github.com/stepan-a/growth/tree/master/routines/introduction][here]]).
This software is proprietary and not free. Here I show how to use Python (and
the matplotlib library) to produce these charts. As an example, I will plot
the evolution of real GDP per capita in France.

Let us start by extracting from =data= (this object is what is called a
=dataframe= and holds all the data) the observations for France. You can
display the observations for France by selecting the corresponding rows as
follows:

#+begin_src python :session plt-ts-en :exports code :tangle no
  data[data["countrycode"]=="FRA"]
#+end_src

#+RESULTS:
#+begin_example
     countrycode country  year       gdppc          pop
5788         FRA  France     1         956         5000
5789         FRA  France  1000         NaN         6500
5790         FRA  France  1280        1321          NaN
5791         FRA  France  1281        1288          NaN
5792         FRA  France  1282        1253          NaN
...          ...     ...   ...         ...          ...
6494         FRA  France  2014       36527  66374.29027
6495         FRA  France  2015       36827  66610.71741
6496         FRA  France  2016       37124   66786.7139
6497         FRA  France  2017  37895.0004  66927.12053
6498         FRA  France  2018  38515.9193   67028.7493

[711 rows x 5 columns]
#+end_example

The extract contains
src_python[:session plt-ts-en]{data[data["countrycode"]=="FRA"].shape[0]} {{{results(=711=)}}} observations, and
GDP per capita is observed annually from 1280 (that is, from the time of
Philip III the Bold, son of Saint Louis) to 1789, then again from 1820
onwards. The file contains no observation between these two dates, thirty
years covering the Revolution, the Empire and the beginning of the
Restoration, and since the corresponding rows are missing rather than filled
with =NaN=, the chart below bridges the gap with a straight line segment.
Population is only observed annually from 1820 onwards. To plot this time
series, one uses the =matplotlib= library (there are other plotting libraries
which might let you obtain charts more to your taste):

#+begin_src python :session plt-ts-en :exports code :tangle gdppc-en.py :results none
  import matplotlib.pyplot as plt
  # Select the data for France from 1280 onwards
  DATA = data[(data["countrycode"]=="FRA") & (data["year"].astype(int)>=1280)][["year","gdppc"]]
  # Convert the data to numeric values (so far they are character strings)
  DATA["year"] = pd.to_numeric(DATA["year"])
  DATA["gdppc"] = pd.to_numeric(DATA["gdppc"])
  # Plot
  DATA.plot(x="year",y="gdppc")
  plt.savefig("gdppc-fr.svg", transparent=True)
#+end_src

#+CAPTION: *GDP per capita in France since 1280.*
#+LABEL: fig:gdppcfr
[[file:gdppc-fr.svg]]

One observes that not much happens until the beginning of the 19th century.
Here the chart shows the level of GDP per capita; it would be more
appropriate to display the natural logarithm of GDP per capita (to smooth out
the explosive behaviour of the variable from the middle of the 19th century
onwards, and possibly to get a better view of the fluctuations of GDP before
1820).