Mark Coleman

Automating Disney

Thu, 29 Apr 2021 22:05:43 -0400

The Problem

My family had the dream of going to Disney for my sister in law birthday in April and after coordinating the house, plane tickets, multiple families, and tickets we had a problem. The park tickets were not purchased at the exact same moment and some of the family was not able to reserve their park! How could we fix this so the family can go to the park together?

The Calendar

We could do the following...

Manually Click Dates
Check Available Parks
Repeat above…forever…hoping to get lucky

This clearly is not something we can do...there must be a way to dig deeper and automate the checking.

Finding The Data

Lets look to see how the calendar actually works.

Inspect page load via developer tools
Check XHR to see if there are any api requests....
Looking through the requests we see calendar

The API

If you view the request in a new window we get some nicely formatted json and the url is human readable.

https://disneyworld.disney.go.com/availability-calendar/api/calendar?segment=tickets&startDate=2021-04-05&endDate=2021-04-30

Thankful we see that the query string has a lot of helpful parameters

startDate
endDate
segment

Decoding Response

Now that we discovered the format of the request next we need to understand the response.

[
  {
    "date": "2021-04-17",
    "availability": "none",
    "parks": []
  },
  {
    "date": "2021-04-18",
    "availability": "partial",
    "parks": [
      "80007838"
    ]
  },
  {
    "date": "2021-04-19",
    "availability": "partial",
    "parks": [
      "80007838"
    ]
  },
  {
    "date": "2021-04-20",
    "availability": "partial",
    "parks": [
      "80007823",
      "80007838"
    ]
  }
]

The good news the json response is fairly straight forward except for some magic numbers.

date
availability
parks list of parks that are available
- Epcot 80007838
- Animal Kingdom 80007823
- Magic Kingdom 80007944
- Hollywood Studios 80007998

We were able to reverse engineer the above magic numbers by clicking around the calendar and recording which numbers showed up compared to what was displayed on the screen.

NodeJs

Now that we had an api, understood the request and response lets use turn this into a small program using NodeJs and 🔗axios. Our program will need to do the following.

Request Data from the API
Parse the response
If availability do…….something.

Once we put this all together we ended up with the following.

const axios = require("axios");

let url =
    "https://disneyworld.disney.go.com/availability-calendar/api/calendar?segment=tickets&startDate=2021-04-17&endDate=2021-04-17";

(async () => {
    try {
        const response = await axios.get(url);

        var epcot = "80007838";
        var animalKingdom = "80007823";
        var magicKingdom = "80007944";
        var hollywood = "80007998";

        var message = "";
        var alert = false;

        response.data.forEach((date) => {
            message += `Date: ${date.date} | Parks:`;
            var parkNames = [];
            date.parks.forEach((park) => {
                if (park === epcot) {
                    parkNames.push("Epcot");
                }
                if (park === animalKingdom) {
                    parkNames.push("Animal Kingdom");
                }
                if (park === magicKingdom) {
                    alert = true;
                    parkNames.push("Magic Kingdom");
                }
                if (park === hollywood) {
                    parkNames.push("Hollywood");
                }
            });
            message += `${parkNames.join()} \n`;
        });
        if (alert) {
            //send alert
        }
    } catch (error) {
        console.log(error.response.body);
    }
})();

As you can see from the above we parsed the response and then captured the parks that are available and only trigger an alert for Magic Kingdom as that was the park the family wanted to go to on the day 2021-04-17

Automation

Now that we had a node program we needed to run this on a schedule to automagically check ever so often to see if any new parks open up. We could run via cron or similar scheduling tools, manually run it, or a server-less approach.

I decided to go with pipedream as it ticked all the boxes.

Stop writing boilerplate code, struggling with authentication and managing infrastructure. Start connecting APIs with code-level control when you need it — and no code when you don't.

Convert to Workflow

It was fairly straightforward to convert the NodeJs program to a pipe dream workflow as it had node support at its core which made the migration easy.

Steps

The first thing we needed to do is define a few steps to help flow together what has to happen at each execution along the way.

Dates

We had a step for date that just returns the date we want to check for the calendar checking step.

async (event, steps) => {
    return {
        date: "2021-04-17"
    }
}

Check API

As you can see this is a fairly easy migration but a few changes happened to make it work in pipedream

pass in the date via the steps object steps.date_to_check.$return_value.date;
return an object so it can be used in future steps

        return {
            alert: alert,
            message: message
        };

async (event, steps) => {
    const axios = require("axios");

    let date = steps.date_to_check.$return_value.date;
    let url =
        `https://disneyworld.disney.go.com/availability-calendar/api/calendar?segment=tickets&startDate=${date}&endDate=${date}`;

    try {
        const response = await axios.get(url);

        var epcot = "80007838";
        var animalKingdom = "80007823";
        var magicKingdom = "80007944";
        var hollywood = "80007998";

        var message = "";
        var alert = false;

        response.data.forEach((date) => {
            message += `Date: ${date.date} | Parks:`;
            var parkNames = [];
            date.parks.forEach((park) => {
                if (park === epcot) {
                    parkNames.push("Epcot");
                }
                if (park === animalKingdom) {
                    parkNames.push("Animal Kingdom");
                }
                if (park === magicKingdom) {
                    alert = true;
                    parkNames.push("Magic Kingdom");
                }
                if (park === hollywood) {
                    parkNames.push("Hollywood");
                }
            });
            message += `${parkNames.join()} \n`;
        });
        return {
            alert: alert,
            message: message
        };
    } catch (error) {
        console.log(error.response.body);
    }
}

Email

As we said above we want to do something this if the alert is triggered which in this workflow happens to be an email. My favorite zero friction email service happens to be sendgrid and good news is pipedream has a built in connector to send grid.

async (event, steps, params, auths) => {
    const axios = require('axios')
    if (steps.check_disney_calendar.$return_value.alert) {
        return await require("@pipedreamhq/platform").axios(this, {
            url: `https://api.sendgrid.com/v3/mail/send`,
            headers: {
                Authorization: `Bearer ${auths.sendgrid.api_key}`,
            },
            method: 'POST',
            data: {
                "personalizations": [{
                    "to": [{
                        "email": params.to_email,
                        "name": params.to_name
                    }],
                    "subject": params.subject
                }],
                "from": {
                    "email": params.from_email,
                    "name": params.from_name
                },
                "reply_to": {
                    "email": params.reply_to_email,
                    "name": params.reply_to_name
                },
                "content": [{
                    "type": params.type,
                    "value": params.value
                }]
            }
        })
    }
}

we modified the built in template slightly to only send the email if steps.check_disney_calendar.$return_value.alert is actually true

Now that we have all of these in place it is time to try it out by finding a date that has availability and use that in the date step.

Testing It Out

We set the workflow to trigger every 10 minutes and as soon as we hit the right condition we got the above email.

Time To Wait...Did it work?

Yes

It actually did work we triggered the check every 10 minutes and as soon as I got the email I resorted to the manual flow of contacting every family member. I could have easily used twilio to make a phone call, maybe that will be in the next iteration.

Push Container to Registry in GitLab

Sun, 28 Jul 2019 11:58:46 -0400

Navigate over to the access token section in your profile.

Scope to api and set your expiration date.

store this key in your favorite password vault as it is not accessible after the generation process

docker login registry.gitlab.com

username: {your username for gitlab}
password: {the access toke you just crated}

build and push your image

cd site && bundle exec jekyll build
cd ..
docker build . -t registry.gitlab.com/markcoleman/vikingmill:1.0 -t registry.gitlab.com/markcoleman/vikingmill:latest
docker push registry.gitlab.com/markcoleman/vikingmill

tagged via a version and also latest

Container Registry

Setting Up a GitLab Pipeline

Sat, 27 Jul 2019 11:38:19 -0400

I have been slowly exploring the free tier of gitlab and today wanted to setup a build in the CI job. The following steps are taken to create a build of a static site via jekyll.

Create `.gitlab-ci.yml`

touch .gitlab-ci.yml

Edit the file

As with any build script I simply put the steps I ran locally and placed them into the yml file.

image: "ruby:2.5"
build:
  script:
    - cd ./site/ && bundle install
    - bundle exec jekyll build
    - zip -r site-build.zip ./_site

note, unlike circleci each script execution will persist directory structure, e.g. cd site now puts you in the site folder for any proceeding script items.

The first Error

$ zip -r site-build.zip ./_site
/bin/bash: line 90: zip: command not found

It appears our image does not contain, zip so we will need to install it via a before_script block.

image: "ruby:2.5"
before_script:
  - apt-get install -y p7zip
build:
  script:
    - cd ./site/ && bundle install
    - ls
    - bundle exec jekyll build
    - zip -r site-build.zip ./_site

Another Error

root@item:/# apt-get install p7zip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package p7zip

I learned that since this is a fresh image you also need to run the update command first.

image: "ruby:2.5"
before_script:
  - apt-get update -qy
  - apt-get -y install zip unzip

Don't forget to add in -y to auto answer the prompt.

The final build `yml` file

image: "ruby:2.5"
before_script:
  - apt-get update -qy
  - apt-get -y install zip unzip
build:
  script:
    - cd ./site/ && bundle install
    - bundle exec jekyll build
    - zip -r site-build.zip ./_site

Artifacts

Now we have all of your pieces in order and our build is successful next up getting an artifact of the build which happens to be the rendered site from jekyll.

Storing the contents of the zip as an artifact

  artifacts:
    name: "viking-mill"
    paths:
      - ./site/site-build.zip
    expire_in: 30 days

note, this new block will reset to the root working folder so you need to specify the full path from the root

The Final `.gitlab-ci.yml`

image: "ruby:2.5"
before_script:
  - apt-get update -qy
  - apt-get -y install zip unzip
build:
  script:
    - cd ./site/ && bundle install
    - bundle exec jekyll build
    - zip -r site-build.zip ./_site
  artifacts:
    name: "viking-mill"
    paths:
      - ./site/site-build.zip
    expire_in: 30 days

Finish

Now we have a build setup inside of gitlab for the static jekyll site