Never miss an update:

  • Hubot: Get notified in Slack on merge conflicts

    This is the final out of three posts where I will explain how to hook up Hubot to Github’s API. The goal of these posts is to get notified in Slack when your pull request becomes unmergeable.

    Read “Hubot: get it running locally in Slack” and “Hubot: listen to Github’s pull request events” to get up to speed and get the code that this post builds upon.

    Connect to Github’s API to check the merge status of a pull request

    Is your pull request event set up and your bot listening to your webhook? Great! Let’s do something with that data.

    Here you can find an example of the data you’ll receive on pull request events. As you can see there’s a mergeable key. This is either null, true, or false. You want to send a notification to Slack when the value is false.

    Hubot has built-in event listener methods: emit and on. With these methods, you can create your own event that Hubot will listen to. You define an event with emit and you listen to it with on.

    You can use these methods to handle errors. Wrap your data assignment in a try/catch block and create an error event if there is one:

      robot.router.post '/hubot/github/:room', (req, res) ->
        room = req.params.room
    
        try
          data = req.body
        catch error
          robot.emit 'error', error

    Then somewhere in your script you can add:

      robot.on 'error', (error) ->
        #process the error

    Now that you have that sorted, you can set the values you need to work with. In this case: the Slack room name to post in and the pull request url, id and status. Add the following above the catch error line:

      pull_request =
        {
          room: room,
          url: data.pull_request.url
          pullId: data.pull_request.number
          pullState: data.pull_request.state
        }

    The room is needed for the Slack notification. The url and number are needed for the GET request you’ll send to Github’s API in order to check the pull request’s merge status. The pull request’s state is needed to filter events on pull requests that are not open. Let’s do that first, because there’s no point in checking for a possible merge conflict in closed ones.

      if (pull_request.pullState == 'open' || pull_request.pullState == 'reopened')
        # do stuff

    Tip: throw a console.log() in your code to see what data you have available and know what you’re working with.

    When someone pushes to a branch that has a pull request, the payload is sent immediately. The result is that the initial value of mergeable in the data object is unknown. The main reason is that it takes a few seconds until the merge status is known because the code has to be compared to the master branch. In order to know if the pull request has a merge conflict, therefore unable to merge with the master branch, you have to check its merge status via the Github API.

    You can use Hubot’s http call methods, that enable you to send GET requests. In this case to the Github API. Let’s create a method that takes care of this and add it at the top in github.coffee , above the export function:

      getRequest = (robot, data, callback) ->
        url = "#{url}/YourRepoName/#{data.repository}/pulls/#{data.pullId}?access_token=#{token}"
    
        robot.http(url)
          .headers('Accept': 'application/rubyon')
          .get() (err, res, body) ->
            callback(err, res, body)
    
      module.exports = (robot) ->

    The method accepts 3 arguments: the robot instance, data and a callback function.

    Notice that the url variable uses data.repository and data.pullId. These come from the pull_request object that you pass as the data argument. There’s also an access_token variable at the end. This is needed in case of a private repository. You can set one here. If your repo is public, you do not have to include the token in the url. What follows is the GET request to that url.

    Create another method underneath called checkMergeStatus:

      checkMergeStatus = (robot, data) ->
        getRequest robot, data, (err, res, body) ->
          try
            response = JSON.parse body
            mergeStatus = response.mergeable
          catch error
            robot.emit 'error', error

    This method uses your getRequest method and stores the mergeable value. Now, if mergeStatus is false, you want to send a notification to your Slack channel. If it is true you do not have to do anything. If it is still unknown, you want to re-check. Add the following to the checkMergeStatus method, inside the try block:

      if (mergeStatus == false)
        # send notification
      else if (mergeStatus == 'unknown')
        setTimeout ->
          checkMergeStatus(robot, data)
        , 1000
      else
        # do something?

    As you can see, if the second condition evaluates to true, the checkMergeStatus will keep calling itself with 1000ms (1sec) intervals, until it changes.

    Send a notification to your Slack channel on a merge conflict

    Here’s where the previously mentioned emit method comes in handy again. You can create a merge_conflict event for when the status is false. Make sure it has the values you need in your Slack notification:

      if (mergeStatus == false)
        robot.emit 'merge_conflict', {
          room: data.room,
          pullTitle: response.title,
          author: response.user.login,
          pullUrl: response.html_url,
          pullId: response.number
        }

    All that you have to do now, is to get Hubot to take action as soon as the merge_conflict event happens. You can let Hubot ‘listen’ to it by using the on method. I moved the code for this to a separate script called merge_conflict_notifier.coffee in the scripts folder:

      module.exports = (robot) ->
    
        robot.on 'merge_conflict', (merge_conflict) ->
          room_id = robot.adapter.client.rtm.dataStore.getChannelByName(merge_conflict.room).id
          message =
            {
              "text": ":no_entry_sign: Merge conflict: <#{merge_conflict.pullUrl}|##{merge_conflict.number} #{merge_conflict.pullTitle}> by #{merge_conflict.author}"
            }
    
          robot.messageRoom room_id, message

    In order to post a message to a room via Slack’s API, you need to have the room’s id. robot.adapter.client.rtm.dataStore.getChannelByName(<your room name>).id will get that for you.

    The message is formatted according to Slack’s requirements. More information about message formatting can be found on their website.

    Keeping track

    But what happens if someone is going to commit several fixes to their branch but the merge conflict does not get resolved right away? You do not want to spam your Slack channel because that is super annoying. Therefore it would be a good idea to keep a list of the pull requests that have a merge conflict and remove them if they are resolved.

    At the top of the github.coffee, add an empty array called unmergeablePulls. Now you can add some code to keep track of them. Add the following above the checkMergeStatus method:

      unmergeablePulls = []
    
      addPullIdToList = (pullId) ->
        return false if pullId in unmergeablePulls
    
        unmergeablePulls.push pullId
        return true
    
      removePullIdFromList = (pullId) ->
        return if pullId == null
    
        indexOfPullId = unmergeablePulls.indexOf(pullId)
        unmergeablePulls.splice(indexOfPullId, pullId)

    Now you can use these methods in the if/else block within the checkMergeStatus method and add the id of the pull request when there’s a merge conflict. When the id has already been added, the notification is skipped. When there is no longer a merge conflict, the id is removed from the list:

      if (mergeStatus == false)
        if addPullIdToList(robot, data.pullId)
          robot.emit 'merge_conflict', {
            room: data.room,
            pullTitle: response.title,
            author: response.user.login,
            pullUrl: response.html_url,
            pullId: response.number
          }
      else if (mergeStatus == 'unknown')
        setTimeout ->
          checkMergeStatus(robot, data)
        , 1000
      else
        removePullIdFromList(data.pullId)

    Hubot has a robot.brain method that lets you store key-values to Redis. This could also be used to keep track of the id’s, but for now this is how I made it work.

    You can deploy your Hubot to Heroku. Hubot provides documentation on how to do that.

    I hope these 3 posts have helped you get an idea of how Hubot works, the advantages of a chatbot in general and how it can enhance your productivity/workflow.

    Happy hacking :)

    Read more
  • Efficient and effective working: a Product Tank meetup with special guest David Allen

    We were proud to host the ProductTank AMS June 22nd meetup about efficient and effective working at Springest.

    As a product-y person, you are usually required to communicate often and with many people. How do you make sure you still have time for “deep work”?

    None other than David Allen, creator of the Getting Things Done Time Management method, opened the evening. He made an entertaining plea for you to not attempt to store all your new ideas in your brain. That thing is already busy enough dealing with the 50.000 conscious and subconscious decisions you make each day. “Your mind is a crappy office.” Tough to argue with.

    http://img.springe.st/product_tank_meetup_springest_june_2017_david_allen.jpg

    David also gave the crowd a sense of how his company has been running on Holacracy –an operating system for organizations– for the past 6 years.

    This served as a nice bridge to Jaap-Willem Mol from VergaderVerkalking (“if you’re not Dutch and you manage to pronounce that, I’ll buy you a beer”). Jaap-Willem talked about the three basic principles that lie central to effective meetings: rules, roles and feedback. He had a good tip: find inspiration for more effective meetings in the Holacracy Tactical & Governance meeting at Springest video produced by VergaderVerkalking.

    http://img.springe.st/product_tank_meetup_springest_june_2017_jaap_willem_mol.jpg

    Dennis Paagman, Product Owner & Developer at Springest, closed the round of presentations. He showed how GTD & Holacracy help us be productive and aligned on our goals, from the whole company down to each circle and individual role. Some concrete examples included quick peeks at our publicly visible & continuously evolving Holacratic structure at roles.springest.com, and an example of our actual current OKRs (Objectives and Key Results).

    http://img.springe.st/product_tank_meetup_springest_june_2017_dennis_paagman.jpg

    ProductTank organizer Iris van de Kieft closed the event by hosting a nice panel session with the three speakers and an engaged audience.

    http://img.springe.st/product_tank_meetup_springest_june_2017_panel.jpg

    Thanks to the audience, the speakers and ProductTank. We love hosting meetups at our Springest HQ (with a view over the IJ river!). For example, we’re actively involved in the Amsterdam & Netherlands communities for Product, Holacracy and Ruby. So, reach out to Dennis if you’re interested! We’re also hiring backend/frontend/full-stack developers, spread the word!

    Photo’s taken by Mark Mulder, Developer at Springest.

    Read more
  • Hubot: listen to Github's pull request events

    This is the second out of three posts where I will explain how to hook up Hubot to Github’s API. The goal of these posts is to get notified in Slack when your pull request becomes unmergeable.

    If you have never worked with Hubot in Slack before, read the first post: “Hubot: get it running locally in Slack”

    Github’s API

    Github offers an extensive API to connect to your repositories and automate your daily development workflow. Their documentation is very detailed and extensive and they have excellent guides to help you get started. For the goal of this post, I’ll focus on Pull Request events.

    Enable Github’s Pull Request events

    As said above, the goal of this project is to get notified in Slack when a merge-conflict occurs. Luckily, Github supports webhooks. That means you can get them to notify you whenever an event, like a push or a commit, happens, which in turn enables you to check the status of your pull request. Read more about all the events at https://developer.github.com/webhooks/

    Send events to a webhook.

    When an event happens on a pull request, Github gathers all relevant data and sends that event (a payload) to the URL of your choosing. This is called a webhook.

    To set this up, go to the settings page of your repository. Select Webhooks. Then click the add webhook button. You have to provide a so called payload URL which is the webhook URL. Since you are running everything locally, you do not yet have a public URL.

    Set up ngrok

    Ngrok is a tool that enables you to expose your local webserver to the internet. Download it here. Unzip it from your bot’s directory. This will install a small executable in your bot’s root directory:

      $ unzip /path/to/ngrok.zip

    Run it on port 8080, which is Hubot’s default port:

      $ ./ngrok http 8080

    You should see something like this:

      ngrok by @inconshreveable
    
      Tunnel Status                 online
      Version                       2.1.3
      Region                        United States (us)
      Web Interface                 http://127.0.0.1:4040
      Forwarding                    http://a3b9e4e6.ngrok.io -> localhost:8080
      Forwarding                    https://a3b9e4e6.ngrok.io -> localhost:8080
    
      Connections                   ttl     opn     rt1     rt5     p50     p90
                                    0       0       0.00    0.00    0.00    0.00

    You can use the http:// URL for now, you can open that in your browser to see a nifty web interface.

    Set up the webhook.

    Copy and paste the entire URL, ending with .io into the Payload URL field of the form. Append /hubot/github/general to the URL. The /hubot part is important because it is a requirement. The next part, in this example /github, can be anything you like and /general is the room’s name in Slack (which you are free to change as well).

    Make sure the content type is set to application/json and skip the secret field for now. The secret is used to protect your endpoint so we know that messages are coming from Github, but for brevity we’ll leave this up to you, the reader.

    Select Let me select individual events. In this case you only want to receive pull request events. Click Add webhook.

    Note: Every time you restart ngrok, you’ll get a new URL. Make sure to update it in your Github repo settings.

    Test the webhook.

    You will see a list of recent payload deliveries underneath your webhook form right after you save it. There is also a red error mark next to the first delivery.

    Now check ngrok in your console. You should see this:

      HTTP Requests
      -------------
    
      POST /                         404 Not Found

    This means 2 things:

    • The request was received by your local server. Yay!
    • The endpoint it’s routed to, doesn’t exist yet.

    Note: If you do not have your bot running, you’ll see a 502 Bad Gateway error.

    Make Hubot listen to the webhook

    In order to get rid of the 404 error, you have to setup an endpoint in your bot that will execute your code when a payload is received. In your favorite editor (I use Vim), create a new coffeescript file called github.coffee inside your bot’s scripts directory.

    First things first, export a function:

      module.exports = (robot) ->

    This is a requirement and part of the anatomy of a Hubot script. The robot parameter is an instance of your bot.

    Create the endpoint:

      robot.router.post '/hubot/github/:room', (req, res) ->
        room = req.params.room
        data = req.body
    
        console.log("== Pull request data received: #{data.pull_request.number}")

    robot.router is a built-in Hubot method. By adding .post you create a route for POST requests. Naturally, if you want to create a route for GET request, you simply call .get .

    The string '/hubot/github/:room' is the webhook URL you set up in your Github repo, where :room is a variable you have to define. The callback, (req, res) is called when a POST request comes in. Save the room name and the JSON object that contains all data in 2 variables: room and data. Then, to check that it works, the pull request’s number is printed out and will show up in your terminal.

    Finally, you want to tell Github you’ve successfully processed the request. You can do that by sending a success response:

      res.send 'OK'

    That’s it! Your Hubot is now listening to pull request events from Github. You can now move on to do stuff with the data you receive

    Read more
  • Hubot: get it running locally in Slack

    This is the first out of three posts where I will explain how we hooked up Hubot to Github’s API. The goal of this is to get notified in Slack when your pull request becomes unmergeable.

    A Springest Hackday idea

    At Springest we hold regular hackdays. The purpose of these days is to learn, have fun together and work on something that is related to Springest, but isn’t part of our daily work-routine. It can be anything from hanging a swing in the office to building the company’s internal Facebook app called Sputr.

    Starting the day, we all share our ideas during stand-up and we get to work. Either solo or in a team.

    ‘Get slack notifications when a pull request becomes unmergeable’ was one of the ideas on our hackday-list that I thought was interesting to work on.

    In this post, I’ll explain how to run Hubot locally in Slack, so you can test-drive your code.

    So what is Hubot, anyway?

    Hubot is a chat bot by Github. It is open source and written in CoffeeScript and Node.js. You can automate processes with Hubot through scripts, or just add some flavor to your team’s culture. At Springest, our bot is called ‘Ingrid’. She notifies us about all sorts of things that happen on the Springest website and we can tell her to do stuff for us, like deploying, share support ticket info or give someone karma. All by messaging in Slack.

    Slack is a messaging app for teams. Messaging is categorized in channels that everyone is free to follow or not. It enables us to easily communicate with each other.

    To install Hubot, you need to follow these instructions:

    Node.js and npm

    Open your console and check if you have installed node.js and npm:

      $ node -v
      # should output node.js version
      $ npm -v
      # should output npm version

    Node.js is a server-side JavaScript environment and npm is a package manager for node programs.

    If nothing exists, install node.js by downloading it here. This will also install npm.

    Make sure you have the latest version of npm installed, by running:

      $ npm install npm -g

    If you have never installed hubot before, you need to install its generator. Otherwise you can skip this step.

      $ npm install -g yo generator-hubot

    Let’s run it!

    Now that you have the generator, add a directory for your bot and create it. I asked my six-year-old son what name he would give a robot, if he would have one. He said ‘Stone’ because robots are hard like stone. So I called it Stone for this example, no questions asked.

    Create your bot (just hit ‘enter’ for all questions) :

      $ mkdir stone
      $ cd stone
      $ yo hubot
    
      ? Owner irisbune <xxxxxxx@gmail.com>
      ? Bot name stone
      ? Description A simple helpful robot for your Company
      ? Bot adapter campfire
         create bin/hubot
         create bin/hubot.cmd
         create Procfile
         create README.md
         create external-scripts.json
         create hubot-scripts.json
         create .gitignore
         create package.json
         create scripts/example.coffee
         create .editorconfig
                           _____________________________
       _____              /                             \
       \    \             |   Self-replication process   |
       |    |    _____    |          complete...         |
       |__\\|   /_____\   \     Good luck with that.    /
         |//+  |[^_/\_]|   /----------------------------
        |   | _|___@@__|__
        +===+/  ///     \_\
         | |_\ /// HUBOT/\\
         |___/\//      /  \\
               \      /   +---+
                \____/    |   |
                 | //|    +===+
                  \//      |xx|
    
      loadDep:mime-db → request ▄ ╢████████████

    Check if it is alive and kicking by running:

      $ bin/hubot

    If all went well, you should now be able to chat with your bot in the console. Type ‘your-bot-name the rules’ and see if you get a reply.

    Move from console to Slack

    Create an account on Slack if you do not have one yet. I’ve created my own personal channel, to test this bot before implementing the code in Ingrid, our company bot. Once you have your own channel set up, go to https:// yourslackchannel .slack.com/apps/A0F7XDU93-hubot and click on the green button, ‘Add Configuration’.

    Add your bot’s username and hit ‘Add Hubot Configuration’.

    Keep this page open, because you need to copy the environment variable in order to run your bot in Slack.

    Slack adapter

    Now that you’ve configured your bot in Slack, and got your token, you have to install the Slack adapter and your bot is setup to run locally:

      $ npm install hubot-slack --save

    Open Slack on your computer and restart your bot with Slack’s environment variable, followed by a call to hubot’s scripts, plus the adapter flag:

      $ HUBOT_SLACK_TOKEN=your-hubot-api-token ./bin/hubot --adapter slack

    Invite your bot to your #random Slack channel (/invite @botname) and test it out by typing @botname pug me

    Congratulations! You are now ready to hook up Github’s pull request events.

    Read more
  • Vim: Buffers, Tabs, Windows & Modes

    This is the third and last post in this Vim series. You might be interested in reading through my previous posts before you continue reading (especially if you are a real beginner):

    In this post we will talk about buffers, tabs, windows and the different modes that you can find in Vim. Let’s jump in!

    Buffers, windows and tabs

    If you are moving to Vim from another editor like SublimeText or Atom, you are used to working with tabs in a certain way. Specifically, a tab represents an open file, and as soon as you close it, it goes away. A web browser follows this same principle as well.

    Vim has a system for tabs too, but it works in a completely different way from what you are used to. I got quite confused by this when I first started with Vim, so don’t panic if that’s the case for you as well. You are not alone!

    In Vim, there are three levels of view abstraction: buffers, windows, and tabs. Let’s look at each of them from the ground up, since it’s the best way to understand the differences in concept and learn how to use them properly.

    Buffers

    A buffer in Vim is an open instance of a file. This means that the file may not be visible on the current screen, but it is saved somewhere in memory.

    Whenever you open a file in Vim, that file gets put into a buffer that will remain in memory until you explicitly delete it with a call to :quit or :bdelete. You can list all buffers currently open within a Vim session by typing :ls.

    Let’s look at some other useful commands:

    • zz - Center the current line within the window
    • zt - Bring the current line to the top of the window
    • zb - Bring the current line to the bottom of the window

    Although files in Vim’s buffer may not be visible at all times, its functionality is analogous to how you use tabs in familiar text editors.

    Windows

    A window in Vim is a viewport onto a single buffer. You can open a new window with :split or :vsplit, including a filename in the call. This opens your file as a new buffer (again, similar to a tab in a traditional editor) and opens a new window to display it.

    This is what a Vim session with multiple windows open (horizontally and vertically) looks like:

    Windows are also referred to as Splits.

    Let’s look at some useful commands:

    • :new [filename] - Open a new window above the current window
    • :vnew [filename] - Open a new window beside the current window
    • :split <filename> - Edit the specified file in new window above the current window
    • :vsplit <filename> - Edit the specified file in a new window beside the current window

    • <Ctrl-w>h,j,k,l - Navigate to the window in the given direction

    Tabs

    Finally, a tab in Vim is a collection of one or more windows. This allows you to group windows in a useful way.

    Let’s look at some related commands:

    • :tabnew - Open a new tab
    • :tabedit <filename> - Edit the file with the provided name in a new tab

    • gt - Go to next tab open
    • gT - Go to previous tab

    • <Ctrl-w>T - Break the current window out to a new tab

    Modes

    Enough about navigation for now. Let’s move on and talk about the different modes that you will find in Vim’s world!

    Vim is a “modal” editor, which means it has various modes that change its behavior in response to your key presses. This modal nature is at the core of Vim’s power, so it’s very important to understand it in order to use Vim in the most efficient way.

    Vim has three different modes: insert, normal and visual. Let’s now look at them one at a time.

    Insert Mode

    When you use other editors like SublimeText or Atom, you’re always working in insert mode. In this mode, characters appear immediately in the buffer as you type them. You can enter insert mode by pressing i in normal mode.

    However, Vim prioritizes moving through a file and making targeted edits, which are done in normal mode.

    Normal Mode

    Normal mode is the default mode Vim starts in. You are expected to be in this mode the most of your time, while using all motions and operations that we saw in the previous post.

    This fits with the idea that we, as developers, spend the majority of our time moving and editing within a document, rather than simply adding long blocks of text.

    Visual Mode

    In more familiar text editors, a block of text can be selected by clicking the mouse and dragging over a number of lines or characters. Vim introduces Visual model, which allows you to reuse all motion commands and operators that there are to manipulate blocks of text.

    Enter Visual mode by pressing v in normal mode. Move the cursor using all the normal motions, and Vim will highlight from where you started to where you move the cursor. You can use a number of keys such as d, x, or c to operate on the visual selection, similar to how these keys would operate in normal mode.

    Sometimes you will need to operate on entire lines. Visual Line Mode turns out to be very useful in these cases, and it can be started by pressing V from normal mode.

    But what about selecting a column of text? Vim’s got you covered too, enter Visual Block Mode by pressing <Ctrl-v> from normal mode. Here is a list of the common visual block operations and their mapping:

    • d, or x - Delete the visual block selection
    • c - Change the visual block selection
    • r - Replace all characters in the block with the next character you type
    • I - Insert text before the block
    • A - Insert text after the block

    Be aware that when you change or add any new text, Vim will only show the change happening in the first line of the block. After you complete the change/insertion and hit <esc>, it will replicate to all lines.

    Stay Normal

    If you only take one thing away from reading this blog post, let it be this: Normal mode is your best friend!

    Avoid staying in insert mode for extended periods of time. And also, don’t move along the file while in insert mode. It might be difficult in the beginning, but once you get used to it, you will see how much faster you become. ;)

    Wrapping up

    This series is a quick introduction to the infinite world of Vim. It is a tool help you get started, but remember that learning Vim is a nonstop continuous process. Stay on it, make it part of your daily life, and every time that you find yourself doing something in a very inefficient way, you know what to do… Go online, look for a friend and ask for help!

    Read more

subscribe via RSS