I wrote a little blog post over at the Sparkfun Blog about how our work on the Johnny-Five inventors kit was guided by our love of the NodeBots community.

I’m very very proud of our work on that project. =)

-Francis

A few weeks ago I started maintaining node serialport after a long hiatus. We hadn’t had a release in about a year and we had some outstanding bugs that I wanted to tackle. I had also introduced some complexity around testing, years ago, that was never removed and seemed to be making it harder to work on the project. Exactly a month since my first beta release we’ve released serialport@2.1.0 which is one of the larger releases we’ve ever had. This includes work from 13 people, including myself.

The Change Log:

• Major refactor, bug fixes and docs improvements thanks to @ecksun, @fivdi, @gfcittolin,@jacobrosenthal, @mhart, @nebrius, @pabigot, @paulkaplan, @reconbot, @rodovich,@rwaldron, @sayanee, @tigoe and everyone who reported and helped debug issues!
• Fix binary paths to confirm with modern standards
• Integration tests on CI’s that support it or for the folks at home with an arduino handy
• Upgrade to nan-2.2.1 for memory leak fixes and node 6 compatibility (still not supported)
• Confirm nw.js and electron compatibility
• Make the outpout of .list consistent between platforms and docs
• Define ambiguous flow control flags and document them
• Fix support systems who provide 0 as a valid file descriptor
• Fix race conditions when opening and closing ports that led to errors while reading and writing while closing or opening the port.
• [unix] Fix a double open bug on unix that would cause opening and closing ports repetitively to error.
• [unix] Listing serialports on linux now include more ports (including bluetooth devices eg./dev/rfcommXX) and have less bugs in the output
• [windows] Remove deprecated BuildCommDCB for windows 10 support
• [windows] Fix a memory leak on windows
• [windows] Fix a 100% cpu and possible hang bug when ports were disconnected on windows.

The change log is extensive but doesn’t tell the whole story. This release started as a bug triage. I saw over 100 issues, some a year and a half old. I started helping people close them. If they were over 4 months old I would close them (usually with a resolution!) and urge people to reopen tickets if the problem was still a problem. In doing this, however, I saw a lot of common trends.

On Linux and OSX we had issues reopening ports. On Windows we had problems detecting disconnections. And a lot of support issues were due to errors that were either not meaningful or were delivered at the wrong time. This was scary! Fortunately, I found a lot of patches for these issues already researched, written, and waiting to be merged. It’s why we have 13 authors this release! I was able to test and merge most of the submitted changes, and that fixed the worst of the bugs. I also added many more errors. You’ll be warned a lot earlier if you’re trying to do something that doesn’t make sense. (Eg, if you open an already-open port, you no longer get a cryptic system level issue, you get a “port is already open” error.)

I have a habit of “stress cleaning”. Ever since I was a child if someone was angry I’d start tidying up. In recent years this has evolved to involve refactoring code. If you’re going to refactor code you need to know what it does and you need to ensure the fundamental behavior doesn’t change. We now have better documentation, more test coverage, and cleaner code then we’ve ever had before. In a few notable cases I’ve kept behaviors that are arguably bugs in order to not break the API. It was painful, but I wanted this release to be widely consumed by anyone currently using serialport, with little fear and no outside change. The more people who get the bug fixes the better.

The next release will get to attack these bad behaviors and hopefully provide a much easier library to work with. I’ve got the unfortunate advantage of studying a year of bug reports to design version 3.0.0. A year of people trying to work around issues that were left unfixed. A year of people hitting the same issue over and over. As one of the maintainers I want to apologize to anyone who’s had issues over the past year, and I want to thank you for documenting and researching your efforts. It’s been a big help, keep it up.

We’ve got a roadmap to 3.0 open and available to comment. I’ve shared my ideas but I’d like yours too. And if you’re looking to work on the project, please check out the backlog label. There’s still much to do.

And Thanks.

-Francis

I made a little presentation, a hack, a spike into the world of the unknown. I wanted to see if I could power a johnny-five robot with redux a powerful state manager for JavaScript commonly used with ReactJS. I could, I did, and I think you should too. The code in this presentation gets a little handwavey. I did get it working but never finished getting both the browser side simulation and the hardware controlling processes to share the same state. That would be awesome. You should try it.

Download the PDF here.
Browse the github source.

Or browse with this browser thingy below. The last slide was the losing screen from missile command where the screen flashes red. It really lit up the room.

Tonight I gave a talk at QueensJS. It was my first time speaking there and it completes my trifecta of “borojs” speaking events. (I’ve now spoken at BrooklynJS, ManhattanJS and QueensJS!) I would have done it sooner but Sara wouldn’t let me until now.

The talk was on ES6 Modules and the brave new world popping up around them. It was also about how and why we tried to migrate PouchDB to use them and failed.

You can find the presentation on github, I used Deckset to present the markdown file and generate a PDF for download.

I was working on one of our projects when I happened to run across some code that set off my rubber ducky detector. It’s a page that displays the days production quantities or displays an estimate if it’s too early in the day to tell. Both cases have very similar views and they share a common interface. Our controllers were too fat and our views knew too much.

Here’s the culprit.

class ProductionRunsController < ApplicationController
  def print_recipes
    @date = date_query
    @production_run = production_run_for_date(@date)
    @projection_run = ProductionRunProjection.new(@date) unless @production_run
  end
end

# view
<h1>Print Recipes <%= @date.strftime("%A %b. %e, %Y") %></h1>
<% if @production_run %>
  <%= render 'production_run', object: @production_run %>
<% else %>
  <%= render 'projection_run', object: @projection_run %>
<% end %>

We have this one view that renders either a projection_run or a production_run depending on if a user has a production run on that date or not. The code works but I have a few problems with it.

• We have 3 instance variables in 1 controller action. One of which will always be nil! We literally have a variable holding nil. This totally breaks Sandi Metz rule of having 1 instance variable per action. Hard

• More importantly, what happens later on down the road if the client we’re building this for wants past_runs to be formatted differently? Continuing this pattern, it would probably look something like this.

class ProductionRunsController < ApplicationController
  def print_recipes
    @date = date_query
    @past_run = past_run_for_date(@date)
    @production_run = production_run_for_date(@date)
    @projection = ProductionRunProjection.new(@date) unless @production_run
  end
end

# view
<h1>Print Recipes <%= @date.strftime("%A %b. %e, %Y") %></h1>
  <%= render 'past_run', object: @past_run if @past_run %>
  <%= render 'production_run', object: @production_run if @production_run %>
  <%= render 'projection_run', object: @projection_run if @projection %>

As the conditions that the client gives get greater and greater, this view will get more ifs and more variables in the action. As a best practice, it is good to have the least amount of logic in the views as possible and this has a lot.

A more elegant solution involves two patterns: the facade pattern and duck typing.

The facade pattern

The facade pattern is pretty much what it sounds like. We’re hiding logic in some sort of object and asking that object to do work for us rather than the controller. Doing this allows us to stick to one variable and please the ever lovely Sandi Metz. A quick refactoring might looking something like this:

class ProductionRunsController < ApplicationController
  def print_recipes
    @facade = ProductionRunFacade.new(@date)
  end
end

class ProductionRunFacade
  attr_reader :date

  def initialize(date)
    @date = date
  end

  def production_run
    @_production_run ||= production_run_for_date(date)
  end

  def projection
    @_projection_run ||= projection_run_for_date(date)
  end

  private

  def production_run_for_date
    #some logic
  end

  def projection_run_for_date
    #some logic
  end
end

# view
<h1>Print Recipes <%= @facade.date.strftime("%A %b. %e, %Y") %></h1>
#stuff omitted
<% if @facade.production_run %>
  <%= render @facade.production_run %>
<% else %>
  <%= render @facade.projection %>
<% end %>

It’s getting there but it could use a bit more love. Let’s move the date logic from the view to the facade.

class ProductionRunFacade
  attr_reader :date

  #code omitted

  def formatted_time
    date.strftime("%A %b. %e, %Y")
  end
end

# view
<h1>Print Recipes <%= @date.formatted_time %></h1>

Here’s a small win! and it’s all about those small wins. We can take this a step farther but first let’s talk about Duck Typing.

Duck Typing

Sandi can explain duck typing better than I can. In her words,

Duck types are public interfaces that are not tied to any specific class. These across-class interfaces add enormous flexibility to your application by replacing costly dependencies on class with more forgiving dependencies on messages.

Duck types objects are chameleons that are defined more by their behavior than by their class. This is how the technique get its name; if an object quacks like a duck and walks like a duck, then its class is immaterial, it’s a duck.

An example that you may all be familiar with is the [] operator.

array = [0,1,2,3,4]
string = "hello world"
array[0] #returns 0
string[0] #returns h
array[1..-1] #returns [1,2,3,4]
string[1..-1] #returns "ello world"

We can call the [] operator an both an array and a string, [] is the quack and both objects are ducks.

Now let’s apply this to our code.

class ProductionRunFacade
  attr_reader :date

  def initialize(date)
    @date = date
  end

  def run
    @_run ||= production_run || projection_run
  end

  def formatted_time
    date.strftime("%A %b. %e, %Y")
  end

  private

  def production_run
    #some logic
  end

  def projection_run
    #some logic
  end
end

# view
<h1>Print Recipes <%= @facade.formatted_time %></h1>
<%= render @facade.run %>
<% end %>

There are a couple things that happened here. First, @production_run and @projection_run became one variable called @run. Secondly, we no longer have any measly ifs in our view anymore, we only call render on @facade.run. What is this magic? Here we are using the power of duck typing and trusting that ProductionRun and ProjectionRun have implemented to_partial_path which render calls. Because of this trust we no longer care what the object is but care about the messages it sends.

With this new architecture, adding new potential runs is made extremely easy. Actually, let’s implement past_runs as well! It’s only a few additional lines of code.

class ProductionRunFacade
  attr_reader :date

  def initialize(date)
    @date = date
  end

  def run
    @_run ||= production_run || projection_run || past_run
  end

  def formatted_time
    date.strftime("%A %b. %e, %Y")
  end

  private

  def past_run
    #some logic
  end

  def production_run
    #some logic
  end

  def projection_run
    #some logic
  end
end

Done. We don’t have to add any more ifs in our view and we don’t have to add any additional instance variables in our controller. Easy as quack.

FactoryGirl the oddly named testing tool for database models is something we use a lot at Wizard Development. If you’re unfamiliar with how it works, I strongly suggest you check it out. Unlike fixtures which loads a bunch of data into your database, factories will create the objects you need with the data preloaded. The difference allows you to only get what you need, allows skipping using the database all together (for much faster and more isolated tests) and a few other great things.

Individual models are fairly straightforward to test. I’ll be using examples with rspec and ActiveRecord.

# Model
class User < ActiveRecord::Base
  scope :admins, -> { where(admins: true) }
  def admin!
    update!(admin: true)
  end
end

# Factory
FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    sequence(:email) { |n| "#{n}#{Faker::Internet.email}" }
    admin false

    trait :as_admin do
      admin true
    end
  end
end

# Test
require 'rails_helper'

describe User do
  let(:user) { build_stubbed(:user) }
  let(:admin) { build_stubbed(:user, :as_admin) }

  describe '#admin!' do
    it 'makes a user an admin' do
      user.admin!
      expect(user.admin?).to eq(true)
    end
    it 'keeps admins in power' do
      admin.admin!
      expect(admin.admin?).to eq(true)
    end
  end

  describe '.admins' do
    it 'returns only the admins' do
      admin = create(:admin)
      create(:user)
      expect(User.admins).to contain_exactly(admin)
    end
  end
end

For the model’s instance functions we used FactoryGirl.build_stubbed a method that creates a model that pretends it’s saved to the database. All validations, and database methods will pretend to work as expected and we’ll be sure to never actually talk to the database, which is significantly faster.

For the scope we have to talk to the database so we create both models and ensure the result only has the admin object. Since only those two objects are needed, that’s all we make.

As your app grows you’ll start needing to test service objects that work with several models at once, and your models themselves will get more complected requiring each other to be in specific states to be valid. It’s going to get difficult to have a single factory properly setup the environment for testing. (If you find it impossible to use factories you need to have a long hard look at your design because it wont ever get easier on it’s own.)

A common situation is when you want “Multitenancy” where your app needs to support users having their own objects. This is very straightforward to support with factories, at first.

class User < ActiveRecord::Base
  has_one :photo
end

class Photo < ActiveRecord::Base
  validates :user, presence: true
end

FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    trait :with_photo do
      photo
    end
  end

  factory :photo do
    title "My cat Kris"
  end
end

# to build a stubbed user with a stubbed photo
FactoryGirl.build_stubbed(:user, :with_photo)

Now you’ll probably want to support a user with many photos. FactoryGirl suggessts using the after and before callbacks for creating the associations. Lets try it

class User < ActiveRecord::Base
  has_many :photos
end

class Photo < ActiveRecord::Base
end

FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    trait :with_photos do
      transient do
        photo_count 2
      end

      after(:create) do |user, evaluator|
        create_list(:photo, evaluator.photo_count, user: user)
      end
    end
  end

  factory :photo do
    title "My cat Kris"
  end
end

# to create a user with 2 photos
FactoryGirl.create(:user, :with_photos)

# When we build_stubbed or build or any of the other methods, we no longer have any photos!
FactoryGirl.build(:user, :with_photo) # no photos!
FactoryGirl.build_stubbed(:user, :with_photos) # no photos!

Since this approach only works with specific methods, you’ll either need to write callbacks for each method or do some magic. I’ll rewrite the factory with some magic.

FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    trait :with_photos do
      transient do
        photo_count 2
      end

      photos do |t|
        photo_count.times.map {
          t.association(:photo, user: t.instance_variable_get(:@instance))
       }
     end
    end
  end

  factory :photo do
    title "My cat Kris"
  end
end

# Now however we want our test data we'll get what we expect!
FactoryGirl.create(:user, :with_photos)
FactoryGirl.build(:user, :with_photo)
FactoryGirl.build_stubbed(:user, :with_photos)

The t that’s passed into the block on photos, I think this is called an evaluator internal to FactoryGirl, but I’m not positive. Names are hard. We’re able to use the t.association to mimic however we called the parent factory. When we are building a factory it uses build() when we’re creating it uses create(). Yay!

I know t.instance_variable_get(:@instance) looks very strange but there doesn’t seem to be another way to get a reference to the parent object to give to the child object. Not all children need their parents, but when they do you need to provide them.

We should also note that we’re using a transient attribute to allow us to customize how many photos get created.

# if we want a ton of photos
FactoryGirl.create(:user, :with_photos, photo_count: 400)

Lets go for an even more complex example.

class User < ActiveRecord::Base
  has_many :photos
  has_one :album
end

class Album < ActiveRecord::Base
  has_many :photos
  validates :user, presence: true
end

class Photo < ActiveRecord::Base
  validates :user, presence: true
end

Photos still belong to users but can now also belong to an album that belongs to a user. Lets also ensure there’s always a user for these objects.

A factory setup could be

FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    trait :with_album do
      album
    end
  end

  factory :photo do
    title "My cat Kris"
    user
  end

  factory :album do
    user
    title "Kitties"
    transient { photo_count 2 }
    photos do |t|
      photo_count.times.map { t.association(:photo) }
    end
  end
end

Lets try this out

user = FactoryGirl.create(:user, :with_album)
user.album.photos.count # 2
user.photos.count # 0 !?!?!?!
User.count # 4 !!!!!

We have a user with an album of other users photos! That’s not what we wanted.

The photo factory was creating it’s own users for it’s photos since we didn’t specify who should own them. Additionally the album factory created a user and then got assigned to the one we created. Lets try again.

FactoryGirl.define do
  factory :user do
    sequence(:name) { |n| "#{Faker::Name.name} #{n}" }
    trait :with_album do
      album { t.association(:album, :with_photos, user: t.instance_variable_get(:@instance))
    end
  end

  factory :photo do
    title "My cat Kris"
    user
  end

  factory :album do
    user
    title "Kitties"
    trait :with_photos do
      transient { photo_count 2 }
      photos do |t|
        photo_count.times.map { t.association(:photo, user: user) }
     end
    end
  end
end

The user now gives itself to the album, the album now gives it’s user to the photos and we always get what we expect. We can create any factory and get a user who owns the photos that were generated and never get more users than we expect.

I think this is too complicated. I’m convinced there are easier ways to do the advanced examples in this blog post. When I find them I’ll happily update this post and a bunch of my factory code. In the meantime I’ll live with slightly complicated factories and enjoy easier testing.

Let me leave you with a with a small spec we include with most projects. It ensures that every factory and trait is valid and can be stubbed. And helps you keep all factories usable with expected results. FactoryGirl.lint has some unexpected creation of models and doesn’t cleanup after itself. If you’re using Foreign Key Constraints you’ll get an added bonus of errors when you accidently create models related to stubbed models.

require 'rails_helper'

FactoryGirl.factories.map(&:name).each do |factory_name|
  describe "#{factory_name} factory" do
    it 'builds valid' do
      model = FactoryGirl.build(factory_name)
      expect(model).to be_valid if model.respond_to?(:valid?)
    end

    it 'builds stubbed' do
      model = FactoryGirl.build_stubbed(factory_name)
      expect(model).to be_valid if model.respond_to?(:valid?)
    end
  end
end

It’s been around six months since I quit my job as a PSI Analyst. For those of you unfamiliar with the terminology, it’s pretty much a glorified title for an Excel technician. I lasted a mere 3 months before the banal, repetitive tasks took their toll on me, slowly killing my sanity with every vlookup I did. There were more than a few reasons why I quit, but most of all, the corporate life wasn’t for me. I couldn’t imagine kissing ass for the rest of my life to get that 3% promotion every year. I couldn’t imagine doing something I didn’t enjoy for the rest of my life. I couldn’t imagine not being challenged… ever. So I quit.

I quit before I had anything lined up, but I knew I wanted to code. Luckily I had extremely supportive friends who had every intention of helping me reach my goal (shoutout to @alexiomota and @Yanyeeli ). They helped me research coding schools and vet the ones that would fit my needs. They answered any question to the best of their ability and helped me figure out what I wanted to do. Who knows what I would be doing if it weren’t for them!

A few weeks and interviews later, I got into the school of my choice, Metis. Unfortunately, after Kaplan acquired Dev Boot Camp, they have decided to streamline Metis as a Data Science school and removed Metis’s Ruby on Rails and UX-Front End Development courses. Regardless, my time at Metis was everything I wanted and more (shoutout to @tabfugnic, @dgalarza, harry (please excuse his lack of twitter), and @thoughtbot ).

Although it’s not a literal bootcamp, there was nothing easy about attending a coding “boot camp.” There were around 5 hours of lecture and 3 hours of practice problems almost every day. On top of that, in order to network, we attended meetups to mingle with other developers and expand our networks. If we weren’t able to follow along with the day’s material, there was studying to be done after class. Worst of all, I lost 3 hours commuting back and forth from school. “Easy” was never a word that came up when describing those 3 months. But if I were to go back in time and had the chance to do it over again, I would do it in a heartbeat. Not because I’m some sort of sick masochist or because I have $12,000 to just throw around. Not because the material is impossible to learn independently but because the experience taught me what I was capable of and, more importantly, because of the amazing people I met.

Having thoughtbotters as my instructors was invaluable. Not only are they passionate developers, but they are regarded by many to be some of the best in their trade. This was like having Sir Lancelot teaching me to fence or having Michael Jordan teaching me to shoot. They were able to answer questions based on previous experiences, teach us best practices, and show us how to write clean code. The best thing of all, though, was that they stayed connected and truly developed a relationship with us students.

Six months after I quit my job and started down this path, I’ve landed my first development job as a developer at Wizard Development and it’s everything I wanted. I have amazing co-workers, I’m challenged every day, I have a voice in my team, and I get 20% time to write this post. I get freedom to do things the way I see fit and get to learn on the job. I’m pretty much living the dream.

Out of everything I’ve learned thus far, the most important thing I’ve learned was just how much I rely on others and to embrace it. Looking back, I would never have gotten to this point without the help of friends I’ve had and the friends I’ve made along the way. Being a developer is great because I’m challenged every day and the things that I can learn are limitless. Best of all, the coding community is unrivaled. I’ve received so much help from others and I want to learn enough so that I can do the same and bring people through the same journey that I did. So if you have any questions about anything feel free to reach out to, eli@wizarddevelopment.com

We recently added an API to The Winnower to support our new WordPress Plugin. It was a great experience in both product and software design. WordPress support is the first of a few great features we’re adding to The Winnower to aid our users in publishing.

It was suprising to me, but these days a lot of academic publishing, if not in journals, is done on blogs. The Winnower provided publishing services to papers which were uploaded via Word or Latex, but people would have to copy their blog published papers from whatever they currently use into Word if they wanted publish them. That wasn’t great.

WordPress.org is one of the largest blogging software platforms. It’s crazy customizable and very easy to use. It was a very good first candidate to support. We figured a WordPress plugin would allow people to keep publishing as they do now, and allow them to get DOI’s and go through the open review processes through The Winnower.

The plugin needed to integrate with the site via some sort of API, and we wanted to adopt the JSON Api spec wich describes a restful JSON interface. Internally, the Winnower’s APIs were a bit of a mess. The site had evolved a lot since the initial structure was created and the technical debt that had accrued caused the APIs to loose some of their elegance. Because of that, they didn’t map cleanly to JSON API, or any other restful API.

We started by “green fielding” the API design to match the needs of the plugin and it’s users. Then we built the API alongside the existing interfaces. We built a bridge between the new and the old by creating some service objects that wrangled the internals into a reasonable shape and interface. Finally, we built out the controllers to handle the particulars of the JSON API requests.

This taught us a lot about what we really needed to publish a paper, and how we could restructure the internals across the site. Thankfully, we already had great integration test coverage for our publishing papers. We’re now able to start moving towards a much better designed app without the risk of breaking everything.

The coolest part of this sprint was that we got a lot of insight into our existing user experience (UX). Much of it existed to accommodate the original design as opposed to the features the users needed. We’re going to have a much easier time improving the experience after the internal restructuring and we’re going to find ourselves with a much better product in the end.

-Francis

PS If you’re interested in programming for The Winnower’s API please let us know! In the meantime we’ll keep building new uses for it. =)