Development¶

This covers loosely how we do big feature changes.

Changes that involve new Python dependencies¶

All python dependencies have an associated hash (or several) that are checked at download time. This ensures malicious code doesn’t sneak in through dependencies being hacked, and also makes sure we always get the exact code we developed against. Changes in dependencies, malicious or not, will set off red flags and require human intervention.

A pip requirement stanza with hashes looks something like this:

Django==1.8.15 \
    --hash=sha256:e2e41aeb4fb757575021621dc28fceb9ad137879ae0b854067f1726d9a772807 \
    --hash=sha256:863e543ac985d5cfbce09213fa30bc7c802cbdf60d6db8b5f9dab41e1341eacd

hash lines can be repeated, and other comments can be added. The stanza is delimited by non-comment lines (such as blank lines or other requirements).

Fortunately we do not need to add or edit those manaully. Using pip-compile-multi we list only our top-level dependencies in requirements/*.in. To add a dependency, put it in the appropriate requirements/*.in file, then compile:

pip-compile-multi -g default

Changes that involve database migrations¶

Any changes to the database (model fields, model field data, adding permissions, …) require a migration.

Running migrations¶

To run migrations, you do:

$ ./manage.py migrate

It’ll perform any migrations that haven’t been performed for all apps.

Creating a schema migration¶

To create a new migration the automatic way:

make your model changes
run:
```
./manage.py makemigrations <app>
```
where <app> is the app name (sumo, wiki, questions, …).
run the migration on your machine:
```
./manage.py migrate
```
run the tests to make sure everything works
add the new migration files to git
commit

Creating a data migration¶

Creating data migrations is pretty straight-forward in most cases.

To create a data migration the automatic way:

run:
```
./manage.py makemigrations --empty <app>
```
where <app> is the app name (sumo, wiki, questions, …).
edit the data migration you just created to do what you need it to do
make sure to add reverse_code arguments to all RunPython operations which undoes the changes
add a module-level docstring explaining what this migration is doing
run the migration forwards and backwards to make sure it works correctly
add the new migration file to git
commit

Data migrations for data in non-kitsune apps¶

If you’re doing a data migration that adds data to an app that’s not part of kitsune, but is instead a library (e.g. django-waffle), then create the data migration in the sumo app and add a dependency to the latest migration in the library app.

For example, this adds a dependency to django-waffle’s initial migration:

class Migration(migrations.Migration):

    dependencies = [
        ...
        ('waffle', '0001_initial'),
        ...
    ]

Changes that involve reindexing¶

With Elastic Search, it takes a while to reindex. We need to be able to reindex without taking down search.

This walks through the workflow for making changes to our Elastic Search code that require reindexing.

Things about non-trivial changes¶

We should roll multiple reindex-requiring changes into megapacks when it makes sense and doesn’t add complexity.
Developers should test changes with recent sumo dumps.

Workflow for making the changes¶

work on the changes in a separate branch (just like everything else we do)
make a pull request
get the pull request reviewed
rebase the changes so they’re in two commits:
1. a stage 1 commit that changes ES_WRITE_INDEXES, updates the mappings and updates the indexing code
2. a stage 2 commit that changes ES_INDEXES, changes ES_WRITE_INDEXES, and changes the search view code
Avoid cosmetic changes that don’t need to be made (e.g. pep-8 fixes, etc.)
push those changes to the same pull request
get those two changes reviewed

Once that’s ok, then that branch should get updated from master, then pushed to stage to get tested.

That branch should not land in master, yet.

Workflow for reviewing changes¶

Go through and do a normal review.

After everything looks good, the developer should rebase the changes so they’re in a stage 1 commit and a stage 2 commit.

At that point:

Verify each commit individually. Make sure the code is correct. Make sure the tests pass. Make sure the site is functional.
Verify that the ES_INDEXES and ES_WRITE_INDEXES settings have the correct values in each commit.

Workflow for pushing changes to stage¶

Don’t land the changes in master, yet!

If you hit problems, deploy the master branch back to the stage server and go back to coding/fixing.

Push the branch you have your changes in to the official mozilla/kitsune remote.
Deploy the stage 1 commit to stage.
Verify that search still works.
Verify that the index settings are correct—look at the ES_INDEXES and ES_WRITE_INDEXES values.
Destructively reindex.
Deploy the stage 2 commit to stage.
Verify that search still works.
Verify that the index settings are correct—look at the ES_INDEXES and ES_WRITE_INDEXES values.
Verify bugs that were fixed with the new search code.

Workflow for pushing those changes to production¶

If we’re also doing a production push, first push next to production and verify that everything is fine. Then continue.

Tell the other sumo devs to hold off on pushing to master branch and deploying. Preferably by email and IRC.
Once you’ve told everyone, land the changes in master.
Deploy the stage 1 commit to production.
Verify that search works.
Destructively reindex to the new write index.
When reindexing is done, push the stage 2 commit to production.
Verify that search works.
Verify bugs that were fixed with the new search code.

Pretty sure this process allows us to back out at any time with minimal downtime.

On the next day¶

If everything is still fine, then merge the special branch into master and delete the old read index.

Announce “STUCK THE LANDING!” after a successful mapping change deployment.