By Nathan L. Walls

  • Kings/Raleigh
  • Garland/Raleigh
  • Duck, Duck Duck, Goose/Raleigh
  • Product/Raleigh

Benfits of "throwaway" scripting

I like listening to concerts from some of my favorite artists like Mogwai, Explosions in the Sky and Hot Chip. Some artists have a definitive place to go for concert recordings, such as Reflecting in the Chrome for Nine Inch Nails.

For most artists, I end-up visiting YouTube and finding a concert and recently, I’ve found a bunch on YouTube:

While watching on YouTube is great, I would like to listen to these concerts through iTunes or on my phone.

I looked up how I get YouTube video converted to audio and found this Meta Filter thread.

I ended up using the following idea, highlighted in this comment:

youtube-dl UUGB7bYBlq8
ffmpeg -i UUGB7bYBlq8.WHATEVER -vn -acodec copy 'Artist -Title I Want.mp4'

Three keys here:

  1. Get the IDs of the videos I wanted to convert from YouTube. I did this manually
  2. Install youtube-dl, which I did through Homebrew
  3. Install ffmpeg, also through Homebrew

While there are plenty of online or graphical tools one could use to convert YouTube videos to audio, the benefit of a command line tool is that I could then use these tools in a couple of Ruby scripts.

A lot of times, writing code involves writing tests and solving a problem through an application. Theoretically, I could have done that here. But, that felt like overkill because, right now, I have eight or so concert videos.

I wrote two scripts to help me. The first is download.rb:

#!/usr/bin/env ruby

file_list = "concerts.md"
files = File.readlines(file_list)

files.each do |file|
  `youtube-dl #{file}`

In the file, concerts.md is in the same directory and just contains a list of YouTube video ids.

Once these were all downloaded, I needed another script to convert the video files to audio files. I also wanted to name the resulting files. I could do both with a simple data structure. Here’s converter.rb:

#!/usr/bin/env ruby

files = [
    source: "Caribou _ Full Set _ Pitchfork Music Festival Paris 2014 _ PitchforkTV-j755GoKSrUE.mp4",
    destination: "Caribou_Pitchfork_Music_Festival_Paris_2014.m4a"
    source: "Four Tet Boiler Room London DJ Set-s_Bn6FlvflM.mp4",
    destination: "Four_Tet_Boiler_Room.m4a"
    source: "Four Tet _ Full Set _ Pitchfork Music Festival Paris 2014 _ PitchforkTV-_z6BXpBDSGY.mp4",
    destination: "Four_Tet_Pitchfork_Music_Festival_Paris_2014.m4a"
    source: "Hot Chip _ Full Set _ Pitchfork Music Festival Paris 2013 _ PitchforkTV-ZbpoozOvKqE.mp4",
    destination: "Hot_Chip_Pitchfork_Music_Festival_Paris_2013.m4a"
    source: "Massive Attack - Live at Melt Music Festival (July 2010)-TqYA7uJUbqE.mp4",
    destination: "Massive_Attack_Live_at_Melt_Music_Festival_2010.m4a"
    source: "Mogwai _ Full Set _ Pitchfork Music Festival Paris 2014 _ PitchforkTV-T2aFPe7mrhg.mp4",
    destination: "Mogwai_Pitchfork_Music_Festival_Paris_2014.m4a"
    source: "massive attack and portishead live bristol-JeAEv1R4wMY.mp4",
    destination: "Massive_Attack_and_Portishead_Live_in_Bristol.m4a"
    source: "Explosions In The Sky Live at AB - Ancienne Belgique (Full concert)-VUuz68zkJu8.mkv",
    destination: "Explosions_in_the_Sky_Live_at_Ancienne_Belgique.m4a"

files.each do |file|
  `ffmpeg -i "#{file.fetch(:source)}" -vn -acodec copy converted/#{file.fetch(:destination)}`

Neither of these two files is doing anything particularly difficult. I’m just running those command line utilties. But, I’m not having to run them repeatedly. I was able to use ls and Vim to get the file names into converter.rb, then regular expressions to coerce the file listing into a data structure. I filled out the :destination keys manually. That felt like a pretty decent balance of effort to automation.

If I use this file much more, I may improve both of these scripts into something more mature. But, without waiting for that to happen, I was able to take care of some very pragmatic automation right now to save me some tedium.

I’ll take that.

Reading and Learning: June 17, 2015

I’ve been busy the last few weeks on a client project that consumed a decent chunk of my evenings recently. I learned a lot on the project, and I’ve been able to include a few links and lessons I used along the way.

Articles and books

You are an educated, successful person capable of abstract thought. A VP doing an SVP’s job. Your office, appointed with decent furniture and a healthy amount…

Paul Ford writes 38,000 in Bloomberg about the business of software. Yes, there are shorter books you’ll read. This has been highly spoken of elsewhere, particularly the online experience and the fact that corrections and clarifications are being handled by GitHub.

For an added treat on your desktop machine, open the article, then scroll to the bottom as quickly as you can.

A perceived benefit of a client-side JS framework is the responsiveness of its interface - updates to the UI are instantaneous. A large amount of application…

One of the knocks on Rails (and Ruby) is that they’re slow. Yes, they’re slower than a compiled language. But here are some thoughts on changing how we write Rails applications to deliver in 100 ms. No, not using a client-side framework. Turbolinks.

I feel it’s time to revisit the web vs. native debate, and concede defeat — or, at least, concede that the web cannot, and should not, compete with native when…

With the link above, these are opposite sides of the same coin. Let’s optimize our web applications accordingly.

Startups focus on speed since they are burning cash every day as they search for product/market fit. But over time code/hardware written/built to validate…

Building fast and making technical tradeoffs builds technical debt. The same thing happens at the organizational level and organizational debt is harder to recognize, more damaging, and harder to pay down.

One of the myths of SaaS is that the products are so good, so easy to use, so quick to deploy … that the product sells itself. Given the…

This week I found myself with the rather strange task of testing that styles had been correctly applied to the DOM. There was a good reason for it, honest.…

I just wrapped a client project recently that involves some user-facing customization. This is the technique I used to ensure the style in the customization was being applied correctly. Very helpful.

Recently I worked on a project that required an admin interface for the business owner to manage records within the application. To this end we considered a few different approaches.

Since the goal for this project was to build an MVP (minimum viable product) it was agreed building a custom solution would be too much work. After spending time reviewing both ActiveAdmin and RailsAdmin, a decision was made to go ahead with ActiveAdmin. We much preferred the ActiveAdmin interface and didn’t like the idea of applying configuration in our models for RailsAdmin.

Notwithstanding, ActiveAdmin does have a few of it’s own drawbacks. The first of which is it’s dependency on using Devise for authentication, which conflicts with our preference for using Clearance.

This article was helpful on a recent client project where I implemented ActiveAdmin using Monban for authentication

Late last week, it was revealed that Facebook has seemingly taken either lazy or apathy to a whole new level. Yes, you can apparently now reply to text…

I’ve spent the past 8 years or so looking at ugly code. This isn’t uncommon in software development but in my case, I’ve been looking at different ugly code…

About two months ago I saw a site called “ Stanford Nerd ” on Product Hunt . Stanford Nerd works by sending a text message from the website to a Stanford…

I was an economics major in college, and I’ve been grateful ever since for the few key concepts it drilled into me: things like opportunity cost, sunk cost, and…

Nolan Lawson Fellow JavaScripters, it’s time to admit it: we have a problem with promises. No, not with promises themselves. Promises, as defined by the A+…

Screencasts and presentations

I watched or attended the following:


I listened to the following:

For some background on what’s going on here, see the first tool sharpening post

Reading and Learning: May 17, 2015

A fair amount of reading and podcast listening since the last entry in mid-April. Since putting together a small tool to pull reading information out of Instapaper, these sorts of posts have become much easier to pull together.

My goal is to automate them further using a Rails application I’m building. The application’s aim is to reduce the friction of recording what technical/technical culture material I’ve read and podcasts I’ve listened to. More on that later.

Here’s the latest log:

Articles and books

I read the following:

I am proud to be one of the 17 founders/authors of the The Agile Manifesto back in 2001. I think it provided a jolt of energy, hope of a better way of doing things, of creating software and making the world work better. It was a pivotal turning point.

But in the 14 years since then, we‘ve lost our way. The word “agile” has become sloganized; meaningless at best, jingoist at worst. We have large swaths of people doing “flaccid agile,” a half-hearted attempt at following a few select software development practices, poorly. We have scads of vocal agile zealots—as per the definition that a zealot is one who redoubles their effort after they’ve forgotten their aim.

At Wiredrive, we do a fair amount of code reviews. I had never done one before I started here so it was a new experience for me. I think it’s a good idea to…

What object-oriented programming advice is this code violating? Note: please just answer this in your head, not in the…

Lately I’ve become increasingly sensitive to how little time I have left to learn new technologies. It’s not that I’m nearing death: I’m not, at least not by…

There it was, right in my inbox: If you go ahead and make those changes, we’ll come on board and send you a check for $75,000. 75 grand was a buncha cash for…

Programmers love to discuss interviewing programmers. And hate to discuss it. Interviewing touches the very heart of human social interaction: It’s a process…

As a developer, you spend 90% of your time on code-related activities like reading and maintaining existing code. With such a large chunk of time spent on these…

The end of the beginning, and the beginning of the end It was January, and the weather was fucking bleak. It’d been freezing and overcast for as long as I could…

Of course Facebook doesn’t “edit” NewsFeed in the same way that a newspaper editor once edited the front page. It’s a very different way. That’s why we’re…

Recruiters are not my most favorite people in the tech eco-system. They’re the people that will contact you on behalf of some client (the company that pays…

This is a simple web application for a group of employees to find out their average salary without exposing any individual salary information to anyone in the group, or storing any private information on the server.

ShellCheck is a static analysis and linting tool for sh/bash scripts. It’s mainly focused on handling typical beginner and intermediate level syntax errors and…

This is a simple web application for a group of employees to find out their average salary without exposing any individual salary information to anyone in the…

Today we are announcing our intent to phase out non-secure HTTP. There’s pretty broad agreement that HTTPS is the way forward for the web.  In recent months,…

When I started on the Firebird team at Bazaarvoice, I was happy to learn that they host their code on GitHub and review and land changes via pull requests. I…

Is software antifragile? I think so. I recently finished the book Antifragile – Things that Gain from Disorder by Nassim Nicholas Taleb. I liked it a lot, and…

I’ll never forget the first time I used the World Wide Web. It was in the early 1990s. I was in America visiting my girlfriend (now wife) at her college in Massachusetts. This was before Mosaic, the first graphical web browser, was released. There were no images on the web, but I was still stunned by the scope of what I experienced. Even back then, the web seemed limitless, without edges. That Encarta CD-ROM sitting next to the computer suddenly seemed pathetically constrained.

In her new book No One Understands You and What To Do About It , Heidi Grant Halvorson tells readers a story about her friend, Tim. When Tim started a new job…

Here is my list of heuristics and rules of thumb for software development that I have found useful over the years: Development 1. Start small, then extend.…

Shannon Liss-Riordan rolls a small black suitcase out of the ugly federal courthouse in San Francisco. She’s smiling, just a fraction. No wonder: She just won a little victory against Uber.

You didn’t mean to end up here. You didn’t even see it coming. It all started with a chance to earn a living doing something you loved. Your dream job. Creating…


I listened to the following:

For some background on what’s going on here, see the first tool sharpening post

Reading and Learning: April 20, 2015

For some background on what’s going on here, see the first tool sharpening post

Oh goodness, this is a big one. I can explain. Work has been busier and I’ve spent more time of late really wanting to not do things manually. Witness all of the TextExpander work I’ve done in previous editions of “Tool Sharpening.”

I simply refused to keep a list of articles read manually, so I waited until I wrote a tool to pull the contents of an RSS feed from Instapaper and write it out as a Markdown file. After completing another home project last week, I had time to write the first iteration of that tool, and so, I was able to quickly pull things I’ve read for the last few weeks easily.

As the tool expands, I expect to be able to make these sorts of posts more frequent and, as such, more digestible.

Articles and books

I read the following:

We’ve been paying closer attention lately to how we use design patterns in our Ruby on Rails work. Decorators have emerged as one pattern that’s helped us keep…

Since leaving SocialChorus I have been doing an odd combination of management consulting and on the ground software design consulting. I have been doing what I…

Rome wasn’t built in a day is one of those adages freely dispensed by motivational posters and chatty grandparents, which makes it just as easy to freely…

Last year in one of my conference talks, I mentioned that I have kids. After my talk, a woman came up to me and asked me how I do it. How I have a full-time job…

My hands were shaking…I could barely breathe I had just finished the first one-one-one coding assessment in my six-month coding bootcamp and it had not gone as…

Let’s say you want to delete a method foo that seems to be dead code. You use a tool like grep to find callers of foo, and there are no results. It’s tempting…

On March 27 The following message was posted on the official GitHub blog: We are currently experiencing the largest DDoS (distributed denial of service) attack…

I’d been meaning to write this up for a while, and now Nat Pryce has written up the 140 character version . I “write” a Call Option when I sell someone the…

Let’s say you have some code whose intent is clear to you but you can see a world where someone else may be confused. Often after writing such code, you realize…

Were software engineer a profession like doctor or lawyer, we’d have a strong and binding set of ethics. I note that the ACM publishes a code of ethics. Here’s…

Brent Simmons has a good post today on ethics for programmers: Were software engineer a profession like doctor or lawyer, we’d have a strong and binding set of…

This checklist is comprised of 48 items you can use to gauge the maturity of your software delivery competency, and form a baseline to measure your future…

I am lucky enough to work with a small team of fantastic engineers who truly care about their customers. If you are not that lucky, this letter is for you to…

The hardest part of advising Ph.D. students is teaching them how to write. Fortunately, I’ve seen patterns emerge over the past couple years. So, I’ve decided…

If you’re using Git, you’re probably using pull requests. They’ve been around in some form or other since the dawn of DVCS. Back before Bitbucket and GitHub…

Screencasts and presentations

I watched or attended the following:


I listened to the following:


Tool Sharpening: March 11, 2015

I’m presenting Intermediate Git on March 31. It’s a one-day, hands-on workshop to build skills from beginner to intermediate with Git on the command line. Cost: $449. You can register here.

Environment + Process tweaks

I separated “Reading and Learning” posts from “Tool Sharpening” posts, as I mentioned I was going to do. These are separate concerns and they’ll be easier to make both types of post separately. My expectation is I’ll have roughly twice as many Reading and Learning posts as Tool Sharpening posts.

Also, with my new gig, I’m a lot more comfortable working on a Pull Request model. Even if I’m the only person working on a project, it’s still good process reinforcement. I’m also using a rebase model. I’ve never cared for merge commits. Conversely, reducing a feature branch to one or two very descriptive commits feels pretty good. I’ll use small commits in the moment when I’m taking small steps to implement a feature. Once I finish the feature, those small steps lose their utility. They become noise. The Pull Request model lets me strip away that noise and tell the larger story about what the commit is doing and why its there.

I also:

  • Adjusted the TextExpander shortcuts I use to generate much of each podcast line to account for wording changes I’ve made
  • Refactored my TextExpander shortcuts for podcasts to create the Markdown link for podcasts with a predictable URL structure, which thankfully, is most of the ones I listen to
    • A further refactoring to do is to transition to a single TextExpander shortcut that calls a Ruby script with which I could retrieve the show notes page and grab the episode title
    • This would allow me to more quickly scrobble the podcasts I listen to, even though there’s not a direct way to get the information out of Overcast.
  • Created some TextExpander shortcuts for fitness and exercise
  • I added some additional Composure shell commands for Git
    • grbc is git rebase --continue
      • Useful for resolving conflicts on a merge or, more likely with the way I work, a rebase before a merge
    • gmt is git mergetool
      • Get me into merge conflict resolution as quickly as possible
    • fbc is git push origin :$1 && git branch -d $1 and allows me to clean-up a feature branch after I’ve merged it to master
      • E.g. fbc nlw-new-feature
      • fbc stands for feature branch cleanup
    • gsu is git branch --set-upstream-to=origin/$1 $1
      • Useful for when I’m working with another developer on a feature branch I started and pushed, but now need to pull changes the other develoer made on the feature branch

Project work

  • Moved Accomplished over to GitHub and fixed up the initial batch of tests
  • Set-up a new Digital Ocean server ($10 credit for using the link) that this blog will eventually be migrated to. Outside of the initial machine creation and user addition, I’ll be doing as much of the set-up and management that I can through Ansible

For some more background on what’s going on here, see the first tool sharpening post.

← Previous