Nushell Example #2
Background
I’ve been working with Nushell as a substitute for both a Unix shell and my usual tools for commandline data manipulation. All of my examples are at work though, so I’m writing up a few demonstrations.
Creating a Printable Schedule from an API-driven site
A local charitable festival has a website but the schedule of events doesn’t lend itself well to printing. I’ll work with the data that drives it and generate a schedule that I can carry.
I’ve already inspected the front-end in Firefox’s web tools and identified the back-end API call. From its Javascript I’ve figured out that ‘797’ means “music”. Explaining the browser debugger and how I figured that out is beyond the scope of this post.
If you want to play with the data yourself, I have a gzipped copy here so you won’t abuse their API. Be nice.
# Get all the music events for the weekend from the API...
let music = (http get "https://opositivefestival.org/wp-admin/admin-ajax.php?action=wcs_get_events_json&content[wcs_type]=797&start=2023-10-05&end=2023-10-08")
# If I were using the dumped data above I'd do this to extract
# just the music events...
let music = (
^gzip -dc ~/repos/misc-demos/nushell/opositive.data.json.gz |
from json |
where ($it.terms.wcs_type.0 | any {|x| $x.id == 797})
)
$music | columns
╭────┬────────────────╮
│ 0 │ title │
│ 1 │ id │
│ 2 │ thumbnail │
│ 3 │ thumbnail_size │
│ 4 │ multiday │
│ 5 │ ending │
│ 6 │ duration │
│ 7 │ terms │
│ 8 │ period │
│ 9 │ excerpt │
│ 10 │ hash │
│ 11 │ visible │
│ 12 │ timestamp │
│ 13 │ last │
│ 14 │ start │
│ 15 │ end │
│ 16 │ future │
│ 17 │ finished │
│ 18 │ permalink │
│ 19 │ buttons │
│ 20 │ meta │
╰────┴────────────────╯Great. After a little exploring I know the fields I need: Name, Start,
End, Location, and a description named “Excerpt”. Start and End are
date and time, we’ve converted them in the previous example and will do the same.
Excerpt is a HTML snippet. I’ll use the command line
web browser lynx to render it as text. The first line is roughly the
same as the Location, the second line is a summary, and the remainder
is a free-form description. I’ll split lines and pull out the second.
let schedule = $music |
each {|r| {
Name: $r.title,
Start: $r.start,
End: $r.end,
Location: ($r.terms.wcs_room.0.name),
Description: ($r.excerpt |
^lynx --dump --force-html -stdin |
str trim |
split row -r '\n' |
get 2 |
str trim)
}
} |
group-by { get Start | format date "%Y-%m-%d" }Let’s see what Friday’s events are:
let $friday = $schedule | get "2023-10-06"
$friday
╭────┬────────────────────────────────────────┬───────────────────────────┬───────────────────────────┬───────────────────────┬────────────────────────────────╮
│ # │ Name │ Start │ End │ Location │ Description │
├────┼────────────────────────────────────────┼───────────────────────────┼───────────────────────────┼───────────────────────┼────────────────────────────────┤
│ 0 │ O+ Kickoff Parade featuring Brasskill │ 2023-10-06T17:45:00+00:00 │ 2023-10-06T18:30:00+00:00 │ Kingston Library │ Lineup at 5pm – Parade at │
│ │ │ │ │ │ 5:45pm │
│ 1 │ Banned Jezebel │ 2023-10-06T19:00:00+00:00 │ 2023-10-06T19:45:00+00:00 │ Keegan Ales │ Funk, Blues, Rock & Soul │
│ 2 │ Molly Tigre │ 2023-10-06T19:00:00+00:00 │ 2023-10-06T19:45:00+00:00 │ SO+mewhere Stage │ Ethopian Funk & Progressive │
│ │ │ │ │ │ Jazz │
│ 3 │ The Goddess Party │ 2023-10-06T19:15:00+00:00 │ 2023-10-06T20:00:00+00:00 │ The Old Dutch Church │ A cult worth joining │
│ 4 │ Reed Foehl │ 2023-10-06T19:30:00+00:00 │ 2023-10-06T20:15:00+00:00 │ Utility Bicycle Works │ Grammy Nominated │
│ │ │ │ │ │ Folk/Americana │
│ 5 │ Brothers Choice │ 2023-10-06T20:00:00+00:00 │ 2023-10-06T20:45:00+00:00 │ SO+mewhere Stage │ Rock and Roll! │
│ 6 │ Pencildive │ 2023-10-06T20:00:00+00:00 │ 2023-10-06T20:45:00+00:00 │ Keegan Ales │ Rock / Punk Power Trio │
│ 7 │ Gail Ann Dorsey │ 2023-10-06T20:15:00+00:00 │ 2023-10-06T21:15:00+00:00 │ The Old Dutch Church │ Legendary Bassist │
│ 8 │ The Kondrat Sisters │ 2023-10-06T20:25:00+00:00 │ 2023-10-06T21:10:00+00:00 │ Utility Bicycle Works │ Harmony Driven Folk │
│ 9 │ Juma Sultan's Aboriginal Music Society │ 2023-10-06T21:00:00+00:00 │ 2023-10-06T22:00:00+00:00 │ SO+mewhere Stage │ World-Beat Garage-Jazz │
│ 10 │ Lulu Lewis │ 2023-10-06T21:00:00+00:00 │ 2023-10-06T22:00:00+00:00 │ Keegan Ales │ Dance / Electronic Rock │
│ 11 │ Bashful │ 2023-10-06T21:20:00+00:00 │ 2023-10-06T22:05:00+00:00 │ Utility Bicycle Works │ Emotionally Engineered │
│ │ │ │ │ │ Nostalgia │
│ 12 │ Carsie Blanton │ 2023-10-06T21:30:00+00:00 │ 2023-10-06T22:30:00+00:00 │ The Old Dutch Church │ Protest Songs & Optimistic │
│ │ │ │ │ │ Singalongs! │
│ 13 │ Peter Boofit │ 2023-10-06T22:00:00+00:00 │ 2023-10-06T22:30:00+00:00 │ Keegan Ales │ Original 8 bit │
│ 14 │ John Swan │ 2023-10-06T22:30:00+00:00 │ 2023-10-06T23:30:00+00:00 │ Keegan Ales │ Dance Music Turntable DJ │
╰────┴────────────────────────────────────────┴───────────────────────────┴───────────────────────────┴───────────────────────┴────────────────────────────────╯Good. Now I can create a new column with just the start and end time and drop the columns Start and End. I’ll render it to HTML.
$friday |
upsert Time {|x| [ ($x.Start | format date "%H:%M"), ($x.End| format date "%H:%M" ) ] | str join " - "} |
reject Start End |
to html out> /tmp/friday.htmlFinally, I’ll use pandoc to convert the HTML table to a PDF with
compact margins and small fonts that I can print.
pandoc -V geometry:landscape -V fontsize=8pt -V geometry:margin=.25in -f html -t pdf friday.html -o friday.pdfIt looks pretty good:
