I have spent many hours riding CTA elevated trains. Most notably from my home on the NW side of Chicago to Loyola University’s Downtown campus — Logan Square on the Blue Line to Washington, walk through the pedestrian tunnel to the Red Line, to Chicago Avenue. After graduation, I got an apartment in Evanston and took the Purple line from Linden in Wilmette to the Merchandise Mart where I worked.

So I took a great interest in an ad for this product:

It’s a map of the CTA lines with LEDs that light to indicate where the trains currently are — in real time. I wanted it. But at $300, it was out of reach. So, I’ll make one for myself.

The blog post is about my efforts to create a proof of concept in software before I consider a hardware solution.

The first step is to source the data. It turns out the CTA has a free, public REST API regarding where its trains are. There are multiple endpoints to the API. The most important to this project is ttpositions that shows the location of every active train on a particular line – Red, Blue, Orange, etc…

http://lapi.transitchicago.com/api/1.0/ttpositions.aspx?rt=red

and returns the following interesting values:

A little explanation will help here. Each active train has a unique identifying number. When a train reaches a terminus, the number is recycled.

Next stop shows the station name at which the train will make its next stop. Even if a train is sitting stopped at some station, it will be the next station down the line that will be returned. A train might also be between two stops.

If the train is just about to pull into a station, Approaching is “yes”.

Latitude and Longitude are the map coordinates of the train.

Heading is the compass direction for the train’s motion.

It’s simple to take these lat/lon data and create a KML (Keyhole Markup Language) file and feed that file into Google Earth Pro. KML files are XML. A KML file will contain XML elements to specify colors, font size, icon, and the position and orientation of the viewpoint — the point in space that determines what you’re looking at, etc.

The most critical element of the KML is a place marker for each train. The place marker looks like this:

<Placemark id="7">
  <name>605 Ridgeland 89°</name>
  <styleUrl>#g</styleUrl>
  <Point id="7">
    <coordinates>-87.79378,41.88699,300</coordinates>
  </Point>
</Placemark>

In this example, the train number, next station name and compass heading are used to label each dot on the map. styleUrl references a previously defined style that specifies the formatting of the place marker. The element coordinates specifies the lon, lat.

My minimally viable product runs a query for each CTA Line, outputs the data to the KML file, and opens the file in Google Earth.

If this is done repeatedly every 15, 30 or 60 seconds, you’ll be able to watch each train move through the system.

MACE, You’re Off the Rails!

A glaring hole in this version is the lack of context. We can’t see the elevated tracks.

The API can again help us with that. Another API endpoint is ttfollow which gives details about a specific train — including its coordinates

http://lapi.transitchicago.com/api/1.0/ttfollow.aspx?runnumber=811

My strategy to plot the train tracks was to find a train that was just leaving a terminus and repeatedly query that run number (train number) and collect lat/lon until that train reached the opposite terminus. That set of points can be coded into a KML file as a PolyLine drawn from point to point.

Sounds simple, but that’s when my frustrations began to grow.

Challenges

Zero^th, The documentation for the API says that when you query a line for all trains, it returns an array of train dictionaries. But if there is only one active train on the line, rather than a one element array, it’s a scalar dictionary. And if no trains are running on the line, the “trains” JSON key doesn’t even exist. This wasn’t a problem earlier until I did the Purple line during non-rush hour and the Yellow (Skokie Swift) line. I found solutions, but it makes the code kludgey.

First, it’s difficult to catch a train at a terminus. It’s not like the train is sitting at the O’Hare terminal, broadcasting its GPS and a station name of O’Hare. The best I could do was randomly catch a train soon after it left a terminus moving toward its next stop – Rosemont. So I had to frequently query the Blue Line hoping to see a Rosemont train HEADING EAST (as opposed to a Rosement train heading west from Cumberland!). Once I saw a train starting a run, I had to take its number and train queries.

OK — annoying, but then, patience is a virtue.

Second, as I was monitoring the progress of a train it got to somewhere near the Belmont station when POOF — It drops off the radar never to return. I have no idea why that train disappeared and I have seen this anomaly multiple times since.

OK — annoying, but…

Third, after I’d collected about 60 data points, I plotted them and saw a most bizarre path.

The train appeared to jump forward then back then forward again. I observed this behavior many times and I didn’t believe this is what the train was actually doing. I concluded there is something wrong with the data being returned from the API. Letters to the API manager have gone unanswered.

I ended up developing a workflow where I could identify these spacetime anomalies and correct them. If you look at the illustration above, if you get rid of point #3 (which is the exact same coordinates of point #7), the data looks realistic and reasonable. I imported the data points into Excel and used conditional formatting to highlight duplicates:

Then I simply deleted the first duplicate (which unformats that dup’s twin) and then moved on to the next “first” duplicate. Once no duplicates are present, I put the datapoints into the KML file to plot the line.

I went through this process for each of the CTA Lines.

The end result of all this is the following graphic:

When the train positions are overlayed onto the colored Lines, you get:

If you repeatedly plot the trains, you get a timelapse of the movement of the trains through the system.

https://youtu.be/C2WNu9fGgoc

This exercise shows the hardware version of this project is entirely possible. A prototype of the Purple Line seems very doable. It will need 9 LEDs for each stop plus 8 more for between-station lights. I want a string of LEDs for Southbound trains and a second string for Northbound trains. That makes 36 LEDs total. The processing flow would be:

Query the Purple line and get its trains  
For each train in AllTrains
   Look at the Next Stop and direction of each train
   If Approaching is Yes, 
      light up the station
   Else
   If Approaching is No, 
      light the between-station-LED just prior to Next Stop
Rinse, Repeat

There is still the issue of time warping, but I no longer care. I’ll just assume that if a train is reported at a given lon/lat, then that’s where it is.

Next Stop

The next step is to build a prototype using LED lights and migrate the software to a Raspberry Pi.

I’m 68.56 years old today. My birthstar is that star which is 68.56 light-years away from Earth. The light produced by that star on my birth date is just reaching Earth now! Later, when I’m older, my birthstar will be different.

I wanted to write a program that would list my birthstar(s). I asked my astrophysist friend where I could find an easily queried, public access, database that contained stellar distances. She suggested the Gaia Archive. https://gea.esac.esa.int/archive/

I soon learned that the Gaia Archive contains a table called gaiadr3.gaia_source. This table contains data such as

I knew that right ascension and declination are important for locating stellar objects. (Who can forget the scene in Contact when Jodie Foster first hears alien signals from space and relays the coordinates of those signals to her team in the control room of the VLA radio telescope. https://youtu.be/5WSqMGJpAZk)

But I couldn’t find anything in the table called distance.

ChatGPT

ChatGPT is an artificial intelligence project that has made a lot of news recently. ChatGPT is currently available for free at https://chat.openai.com. I thought I’d use it to help me figure out if the Gaia Archive can tell me my birthstar.

When you launch a ChatGPT session, called a chat, you start with a simple text box. I started my chat with

me: how can I use the Gaia archive to give me a list of stars that are a specific distance away

(The responses from ChatGPT can be verbose, so I’ll redact some of them.)

ChatGPT: …you can perform a query using the Gaia archive’s query interface. The query would likely involve specifying a distance range and using the parallax measurement to calculate the distance to each star. …you can use “ADQL” (Archive Data Query Language) which is similar to SQL…

That sounds promising. I know SQL and I can use parallax to get the distance.

me: how do you compute distance from parallax

ChatGPT: Distance (in parsecs) = 1 / (parallax angle in arcseconds)

That’s a pretty easy formula. I’ll be able to program that into other applications related to astronomy. Now to get a list of stars and their parallax values.

me: write a Gaia archive adql query to list stars that are 69 light years away

I kid you not. I asked ChatGPT to write an ADQL query that I could just paste into the web page that ChatGPT referred to in the first answer. And ChatGPT did it.

ChatGPT: Here is an example of an ADQL query that can be used to list stars that are 69 light years away using the Gaia archive:

SELECT 
    source_id, ra, dec, parallax
FROM 
    gaia_dr3.gaia_source
WHERE 
    parallax >= (3.08567758e-05 - 0.00005) AND
    parallax <= (3.08567758e-05 + 0.00005)

I pasted the code into Gaia’s query interface and got syntax errors. What followed was some back and forth with ChatGPT regarding its bogus advice.

me: that query has unresolved identifiers
it: I apologize for any confusion, you are correct that the query I provided in my previous response would not work as written…
me: the query you gave me should use gaiadr3.gaia_source as the from parameter
it: I apologize for the mistake in my previous response…

Once ChatGPT’s syntax errors were corrected, the query ran for 30 minutes and returned more than 3,000,000 stars that are (supposedly) between 60 and 70 light years away. I kept narrowing down the range and eventually got the results down to 37,000 stars. I don’t know if that’s the correct number so I tried to determine the parallax measurement for a 69 light-year distant star using paper and pencil.

It turns out that while parallax is normally measured in arcseconds, Gaia records milli-arcseconds. ChatGPT didn’t seem to know that. I eventually came up with the correct values to plug into a Gaia query.

REST API

The queries up to this point were very web-interactive with a lot of keyboard and mouse clicks. I wanted to use a command line tool. So I asked,

me: does the Gaia archive have a REST API

ChatGPT: Gaia archive does have a REST API… Currently, the main endpoint of the Gaia archive REST API is: https://gea.esac.esa.int/tap-server/tap

Reading the REST documentation, there’s a Python module that can be installed to make queries easy. I installed it and asked

me: write a python to query the Gaia archive that lists ra, dec and parallax for stars that are 69 light years distant and use the astroquery.gaia module

ChatGPT:

from astroquery.gaia import Gaia

# Define the query
query = """
SELECT ra, dec, parallax
FROM gaiadr2.gaia_source
WHERE 1 / parallax * 0.001 * 3.26 = 69
"""

# Execute the query
job = Gaia.launch_job_async(query)
result = job.get_results()

# Print the result
print(result)

AND IT WORKS! I threw some bells and whistles into the Python and can now get anyone’s birthstar by supplying a birthdate on the command line.

> p3 birthstar.py -h
birthstar.py [-b:][-n][-h]
-b birthdate yyyymmdd  default: 19540627
-n Northern Hemisphere default: False
-h print usage

> p3 birthstar.py -b 19570411 -n
The person was born 4/11/1957
They are 65.77 years old today
Looking for stars 65.76986301369857 light-years distant (+/- 7 light-days)
Target parallax is 49.5906
Only stars in the Northern hemisphere will be listed.
Here's the Gaia ADQL query

SELECT
    source_id, ra, dec, parallax
FROM 
    gaiadr3.gaia_source_lite
WHERE 
    parallax <= 49.60507969737007 AND 
    parallax >= 49.5761590662218 
    AND dec > 0
ORDER BY parallax DESC

Output saved to: 1674002905756O-result.csv

source_id	        ra         dec	   parallax
1858219151114880000  312.54398   29.38393  49.59437
3270079526697710000   56.77948    1.64475  49.58366
3914019231742220000  170.93122    8.56434  49.58089

Conclusion

ChatGPT was an absolute wonder to work with. It understood what I wanted. And despite the misinformation it gave me and its use of magic numbers in its code, I don’t think I could have finished this project as quickly as I did without consulting it.

Adafruit.com has a large selection of electronic components to buy. They also have a very active YouTube channel, a large number of Learning Guides, and software libraries to control all that hardware. Adafruit’s PyPortal dev board is a wi-fi enabled device with a small, color, touch-enabled screen. The PyPortal is programmed using CircuitPython and Adafruit has published several learning guides for it.

Example projects

Here are just a few of the PyPortal projects that I’ve downloaded and modified:

1. 
   Event Countdown Timer
   Modification: A vacation countdown timer that cycles through graphics representing my next four vacation destinations and the number of days until that vacation begins.
2. 
   ISS Tracker
   Modification: Lengthened the satellite’s trail to show the entire orbit just completed.
3. 
   Hurricane Tracker
   Modification: Added trails to show where the hurricanes has been, not just where they are now.
4. 
   Cleveland Museum of Art Display Frame
   Modification: wait for it…

These applications all have one thing in common. They all reach out to the Internet to grab data and then display that data on the screen. When multiple applications follow the same general pattern, then a lot of effort can be saved by using a well written software library — and Adafruit has written one for the PyPortal.

Example code

To get a feel for the PyPortal’s power and ease-of-use, let’s look at the Quote of the Day project. When you point your browser to https://www.adafruit.com/api/quotes.php, you’ll get back something like this:

[
 {
 "text": "Somewhere, something incredible is waiting to be known.", 
 "author": "Sharon Begley",
 }
]

This JSON response from the web site has two pieces of text that we’d like to display on the screen – the quote and the author’s name. To do that using the Adafruit PyPortal Library, the program must create a PyPortal object and initialize it with all the relevant information:

portal = PyPortal(
         url='https://www.adafruit.com/api/quotes.php',
         json_path=([0, "text"], 
                    [0, "author"]),
         text_position=((20, 120),  # screen location for quote 
                        ( 5, 210)), # screen location for author 
         text_color=(0xFFFFFF,      # quote text color 
                     0x8080FF),     # author text color 
         text_wrap=(35,             # characters to wrap for quote 
                     0),            # no wrap for author 
         text_maxlen=(180,          # max length for quote
                       30),         # max length for author
         text_font="/fonts/Arial-ItalicMT-17.bdf",
  )
while True:
     portal.fetch()
     time.sleep(60)

The loop refreshes the screen with a new quote every 60 seconds using the PyPortal’s fetch() method.

The PyPortal library does all the heavy lifting of connecting to your LAN, browsing to the site’s url, converting the returned JSON into a Python dictionary, and displaying the text on screen. Loads of applications follow this same fetch/display model — current weather data, current number of “likes” on my latest YouTube post(s), latest bitcoin price, local gas prices, current DuPage County covid case count, etc.

Displaying Images

The Cleveland Museum of Art Display Frame Learning Guide follows the same general pattern, but includes the the download and display of an image as well as text. (All the previous examples use a static background image.). Let’s see how PyPortal library handles dynamic images. Here’s the new code:

pyportal = PyPortal(
       json_path=["data", 0, "title"],
       text_position=(4, 231),
       text_color=0xFFFFFF,
       text_font="/fonts/OpenSans-9.bdf",                    
       
       image_json_path=["data", 0, "images", "web", "url"],
       image_dim_json_path=(["data",0,"images","web","width"],
                            ["data",0,"images","web","height"])
       image_resize=(320, 225),
       image_position=(0, 0),
 )

The text parameters are still there. So what’s different? For one thing, there is no URL. We’ll deal with that later. And there are new parameters to deal with the image:

• image_json_path – points to the URL of the downloadable image
• image_dim_json_path – points to the dimensions of the downloadable image
• image_resize – fit the image into these dimensions
• image_position – indicates where on the screen to place the resized image

Now, what about the URL for the Museum? In the Quote project, the URL never changed. Each time you .fetch()’d that URL, a different quote was returned. But for the Museum project, you have to provide a different URL for each piece of art in the Museum’s collection. There are 31954 pieces in the Museum collection. The URL for the 54th piece would include an &skip=53 parameter as part of the URL. The main loop below will do that — starting with the 1st piece and sequentially displaying all the pieces.

skipcount=0
while True:
   pyportal.fetch(f'http://.....&skip={skipcount}')
   skipcount+=1
   time.sleep(60)

Art Institute of Chicago (ARTIC)

Having recently visited the Art Institute of Chicago, I got to thinking — can I modify the CMA project to display the art from the ARTIC. Does the ARTIC even have a website like the CMA? YES it does – https://www.artic.edu/open-access/public-api! So I started coding up the PyPortal parameters and ran into a BIG roadblock. The ARTIC database does not list the full path to a piece of art’s downloadable image. The URL must be constructed from several bits of data. Here’s a highly redacted JSON response for a specific piece of ARTIC art:

{
  "data": [
    {      
      "thumbnail": {
          "width": 3000,
          "height": 1502
      },
      "image_id": "cb34b0a8-bc51-d063-aab1-47c7debf3a7b",
      "title": "Ballet at the Paris Opéra"
    }
  ],
  "config": {
    "iiif_url": "https://www.artic.edu/iiif/2",
  }
}

The title of the artwork is straightforward.

The image’s URL can be computed with the following code:

https://www.artic.edu/iiif/2/cb34b0a8-bc51-d063-aab1-47c7debf3a7b/full/!320,240/0/default.jpg

but that doesn’t fit the PyPortal library’s assumption that the JSON will contain the downloadable image’s URL in a single key:value pair.

Open Source

Then at 3:00 AM, I had the idea of editing the PyPortal Library Open Source. So I downloaded the adafruit_ciruitpython_pyportal/adafruit_pyportal library from Github and started searching through the code. I found the right spot in the fetch() method, inserted a patch to build the image url and add that URL back into the JSON using a new key named ‘artic_image_URL’. The PyPortal has no idea that the image_url key is something I created and slipped in rather than coming from the Museum.

I also had to add code to stuff the image’s dimensions into the JSON. I started with [“thumbnail”][“width”] and [“thumbnail”][“height”] from the ARTIC JSON. These are the dimensions of the master, thumbnail image. But I requested !320,240 in the image_url which means the ARTIC would send me an image that fits within a 320×240 bounding box. The actual, downloaded image size might be 166×240 or 320×185 — to preserve the aspect ratio. So I had to compute the dimensions that the PyPortal library was actually going to encounter.

Here’s the complete patch that I made to the Adafruit library:

            # Build the image URL
            PREFIX = json_out["config"]["iiif_url"] + "/"
            IMAGE = json_out["data"][0]["image_id"]
            OPTIONS = '/full/!320,240/0/default.jpg'
            json_out["artic_image_URL"] = PREFIX + IMAGE + OPTIONS
            
            # Compute the aspect of the Library's artwork
            image_Width = json_out['data'][0]['thumbnail']['width']
            image_Height = json_out['data'][0]['thumbnail']['height']
            aspect = image_Width / image_Height

            # Resize the artwork to fit in a 320 x 240 frame
            if aspect > 320.0/240.0:
                 thumb_width = 320
                 thumb_height = int(320 * image_Height / image_Width)
            elif aspect < 320.0/240.0:
                 thumb_height = 240
                 thumb_width = int(240 * image_Width / image_Height)
            else:
                 thumb_width = 320
                 thumb_height = 240
            json_out["thumb_width"] = thumb_width
            json_out["thumb_height"] =  thumb_height

And here is the PyPortal code that retrieves ARTIC images:

portal = PyPortal(image_json_path=["artic_image_URL"],
                  image_dim_json_path=(["thumbnail_width"],
                                       ["thumbnail_height"]),
                 ...)

To get the PyPortal device to run my patched code instead of the original, “compiled” library code, I simply had to replace the /lib/adafruit_circuitpython_pyportal/__init__.mpy file with my modified __init__.py. My .py file happily coexists along side of all the “compiled byte code” .mpy library files.

And this is what appears on the PyPortal:

etc.

Downsides

My solution isn’t flawless. For some reason, landscape oriented thumbnails are not getting resized correctly, but portrait oriented thumbnails are. Secondly, my patch isn’t robust. If I try to run the Hurricane Tracker project and my patch is in play, then the app will crash. I’ve created a Github issue asking that a JSON transform function be created to allow pre-processing of the JSON. I’m waiting to see if anyone takes an interest. https://github.com/adafruit/Adafruit_CircuitPython_PyPortal/issues/126 if you want to “Like and Subscribe” the issue.