@@ -3,10 +3,12 @@ Hacking on OpenShift

 ## Building a Release

-To build an OpenShift release you run the `hack/build-release.sh` script on a system with Docker, which

-will create a build environment image and then execute a cross platform Go build within it. The build

-output will be copied to `_output/releases` as a set of tars containing each version. It will also build

-the `openshift/origin-base` image which is the common parent image for all OpenShift Docker images.

+To build an OpenShift release you run the `hack/build-release.sh` script on a

+system with Docker, which will create a build environment image and then

+execute a cross platform Go build within it. The build output will be copied

+to `_output/releases` as a set of tars containing each version. It will also

+build the `openshift/origin-base` image which is the common parent image for all

+OpenShift Docker images.

     $ make release

@@ -20,38 +22,46 @@ Once a release has been created, it can be pushed:

     $ hack/push-release.sh

-To cut an official tag release, we generally use the images built by [ci.openshift.redhat.com](https://ci.openshift.redhat.com)

+To cut an official tag release, we generally use the images built by

+[ci.openshift.redhat.com](https://ci.openshift.redhat.com)

 under the devenv_ami job.

 1. Create a new git tag `git tag vX.X.X -a -m "vX.X.X" HEAD`

-2. Push the tag to GitHub `git push origin --tags` where `origin` is `github.com/openshift/origin.git`

+2. Push the tag to GitHub `git push origin --tags` where `origin` is

+`github.com/openshift/origin.git`

 3. Run the "devenv_ami" job

-4. Once the images are pushed to the repository, run `OS_PUSH_TAG="vX.X.X" hack/push-release.sh`. Your tag must match the Git tag.

+4. Once the images are pushed to the repository, run `OS_PUSH_TAG="vX.X.X"

+hack/push-release.sh`. Your tag must match the Git tag.

 5. Upload the binary artifacts generated by that build to GitHub release page

-6. Send an email to the dev list, including the important changes prior to the release.

+6. Send an email to the dev list, including the important changes prior to the

+release.

 We generally cut a release before disruptive changes land.

 ## Test Suites

-OpenShift uses three levels of testing - unit tests, integration test, and end-to-end tests (much

-like Kubernetes).

+OpenShift uses three levels of testing - unit tests, integration test, and

+end-to-end tests (much like Kubernetes).

 ### Unit tests

-Unit tests follow standard Go conventions and are intended to test the behavior and output of a

-single package in isolation. All code is expected to be easily testable with mock interfaces and

-stubs, and when they are not it usually means that there's a missing interface or abstraction in the

-code. A unit test should focus on testing that branches and error conditions are properly returned

-and that the interface and code flows work as described. Unit tests can depend on other packages but

-should not depend on other components (an API test should not be writing to etcd).

+Unit tests follow standard Go conventions and are intended to test the behavior

+and output of a single package in isolation. All code is expected to be easily

+testable with mock interfaces and stubs, and when they are not it usually means

+that there's a missing interface or abstraction in the code. A unit test should

+focus on testing that branches and error conditions are properly returned and

+that the interface and code flows work as described. Unit tests can depend

+on other packages but should not depend on other components (an API test should

+not be writing to etcd).

-The unit tests for an entire package should not take more than 0.5s to run, and if they do, are

-probably not really unit tests or need to be rewritten to avoid sleeps or pauses. Coverage on a unit

-test should be above 70% unless the units are a special case.

+The unit tests for an entire package should not take more than 0.5s to run, and

+if they do, are probably not really unit tests or need to be rewritten to avoid

+sleeps or pauses. Coverage on a unit test should be above 70% unless the units

+are a special case.

-See `pkg/template/generator` for examples of unit tests. Unit tests should follow Go conventions.

+See `pkg/template/generator` for examples of unit tests. Unit tests should

+follow Go conventions.

 Run the unit tests with:

@@ -77,7 +87,8 @@ To run all tests with verbose output:

     $ hack/test-go.sh -v

-To change the timeout for individual unit tests, which defaults to one minute, use:

+To change the timeout for individual unit tests, which defaults to one minute,

+use:

     $ TIMEOUT=<timeout> hack/test-go.sh

@@ -89,7 +100,8 @@ To run unit test for an individual kubernetes package:

     $ hack/test-go.sh vendor/k8s.io/kubernetes/examples

-To change the coverage mode, which is `-cover -covermode=atomic` by default, use:

+To change the coverage mode, which is `-cover -covermode=atomic` by default,

+use:

     $ COVERAGE_SPEC="<some coverage specification>" hack/test-go.sh

@@ -108,71 +120,84 @@ report should be stored. For example:

 After that you can open `/path/to/dir/coverage.html` in the browser.

-To generate a jUnit XML report from the output of the tests, and see a summary of the test output

-instead of the full test output, use:

+To generate a jUnit XML report from the output of the tests, and see a summary

+of the test output instead of the full test output, use:

     $ JUNIT_REPORT=true hack/test-go.sh

-`hack/test-go.sh` cannot generate jUnit XML and a coverage report for all packages at once. If you

-require both, you must call `hack/test-go.sh` twice.

+`hack/test-go.sh` cannot generate jUnit XML and a coverage report for all

+packages at once. If you require both, you must call `hack/test-go.sh` twice.

 ### Integration tests

-Integration tests cover multiple components acting together (generally, 2 or 3). These tests should

-focus on ensuring that naturally related components work correctly.  They should not be extensively

-testing branches or error conditions inside packages (that's what unit tests do), but they should

-validate that important success and error paths work across layers (especially when errors are being

-converted from lower level errors). Integration tests should not be testing details of the

-inter-component connections - API tests should not test that the JSON serialized to the wire is

-correctly converted back and forth (unit test responsibility), but they should test that those

-connections have the expected outcomes. The underlying goal of integration tests is to wire together

-the most important components in isolation. Integration tests should be as fast as possible in order

-to enable them to be run repeatedly during testing.  Integration tests that take longer than 0.5s

-are probably trying to test too much together and should be reorganized into separate tests.

-Integration tests should generally be written so that they are starting from a clean slate, but if

-that involves costly setup those components should be tested in isolation.

-

-We break integration tests into two categories, those that use Docker and those that do not.  In

-general, high-level components that depend on the behavior of code running inside a Docker container

-should have at least one or two integration tests that test all the way down to Docker, but those

-should be part of their own test suite.  Testing the API and high level API functions should

-generally not depend on calling into Docker. They are denoted by special test tags and should be in

-their own files so we can selectively build them.

-

-All integration tests are located under `test/integration/*`. All integration tests must set the

-`integration` build tag at the top of their source file, and also declare whether they need etcd

-with the `etcd` build tag and whether they need Docker with the `docker` build tag. For

-special function sets please create sub directories like `test/integration/deployimages`.

+Integration tests cover multiple components acting together (generally, 2 or

+3). These tests should focus on ensuring that naturally related components work

+correctly.  They should not be extensively testing branches or error conditions

+inside packages (that's what unit tests do), but they should validate that

+important success and error paths work across layers (especially when errors

+are being converted from lower level errors). Integration tests should not be

+testing details of the inter-component connections - API tests should not test

+that the JSON serialized to the wire is correctly converted back and forth (unit test

+responsibility), but they should test that those connections have the expected

+outcomes. The underlying goal of integration tests is to wire together the most

+important components in isolation. Integration tests should be as fast as possible

+in order to enable them to be run repeatedly during testing.  Integration tests

+that take longer than 0.5s are probably trying to test too much together and

+should be reorganized into separate tests.  Integration tests should generally

+be written so that they are starting from a clean slate, but if that involves

+costly setup those components should be tested in isolation.

+

+We break integration tests into two categories, those that use Docker and those

+that do not.  In general, high-level components that depend on the behavior of code

+running inside a Docker container should have at least one or two integration tests

+that test all the way down to Docker, but those should be part of their own

+test suite.  Testing the API and high level API functions should generally

+not depend on calling into Docker. They are denoted by special test tags and

+should be in their own files so we can selectively build them.

+

+All integration tests are located under `test/integration/*`. All integration

+tests must set the `integration` build tag at the top of their source file,

+and also declare whether they need etcd with the `etcd` build tag and whether

+they need Docker with the `docker` build tag. For special function sets please

+create sub directories like `test/integration/deployimages`.

 Run the integration tests with:

     $ hack/test-integration.sh

-The script launches an instance of etcd and then invokes the integration tests. If you need to

-execute a subset of integration tests, run:

+The script launches an instance of etcd and then invokes the integration tests.

+If you need to execute a subset of integration tests, run:

     $ hack/test-integration.sh <regex>

-Where `<regex>` is some regular expression that matches the names of all of the tests you want to run.

-The regular expression is passed into `grep -E`, so ensure that the syntax or features you use are supported.

-The default regular expression used is `Test`, which matches all tests.

+Where `<regex>` is some regular expression that matches the names of all

+of the tests you want to run. The regular expression is passed into `grep -E`,

+so ensure that the syntax or features you use are supported. The default

+regular expression used

+is `Test`, which matches all tests.

-Each integration function is executed in its own process so that it cleanly shuts down any background

-goroutines. You will not be able to run more than a single test within a single process.

+Each integration function is executed in its own process so that it cleanly

+shuts down any background

+goroutines. You will not be able to run more than a single test within a single

+process.

-There is a CLI integration test suite which covers general non-Docker functionality of the CLI tool

+There is a CLI integration test suite which covers general non-Docker

+functionality of the CLI tool

 working against the API. Run it with:

     $ hack/test-cmd.sh

-This suite comprises many smaller suites, which are found under `test/cmd` and can be run individually by

-specifying a regex filter, passed through `grep -E` like with integration tests above:

+This suite comprises many smaller suites, which are found under `test/cmd` and

+can be run individually by specifying a regex filter, passed through `grep -E` 

+like with integration tests above:

     $ hack/test-cmd.sh <regex>

 During development, you can run a file `test/cmd/*.sh` directly to test against

-a running server. This can speed up the feedback loop considerably. All `test/cmd/*` tests are expected

-to be executable repeatedly - please file bugs if a test needs cleanup before running.

+a running server. This can speed up the feedback loop considerably. All

+`test/cmd/*` tests are expected

+to be executable repeatedly - please file bugs if a test needs cleanup before

+running.

 For example, start the OpenShift server, create a "test" project, and then run

 `oc new-app` tests against the server:

@@ -180,20 +205,24 @@ For example, start the OpenShift server, create a "test" project, and then run

     $ oc new-project test

     $ test/cmd/newapp.sh

-In order to run the suite, generate a jUnit XML report, and see a summary of the test suite, use:

+In order to run the suite, generate a jUnit XML report, and see a summary of

+the test suite, use:

     $ JUNIT_REPORT='true' hack/test-cmd.sh

 ### End-to-End (e2e) and Extended Tests

-The final test category is end to end tests (e2e) which should verify a long set of flows in the

-product as a user would see them.  Two e2e tests should not overlap more than 10% of function, and

-are not intended to test error conditions in detail. The project examples should be driven by e2e

-tests. e2e tests can also test external components working together.

+The final test category is end to end tests (e2e) which should verify a long

+set of flows in the product as a user would see them.  Two e2e tests should not

+overlap more than 10% of function, and are not intended to test error conditions

+in detail. The project

+examples should be driven by e2e tests. e2e tests can also test external

+components working together.

-The end-to-end suite is currently implemented primarily in Bash, but will be folded into the extended

-suite (located in test/extended) over time. The extended suite is closer to the upstream Kubernetes

-e2e suite and tests the full behavior of a running system.

+The end-to-end suite is currently implemented primarily in Bash, but will be

+folded into the extended suite (located in test/extended) over time.

+The extended suite is closer to the upstream Kubernetes e2e suite and 

+tests the full behavior of a running system.

 Run the end to end tests with:

@@ -203,25 +232,30 @@ Run the extended tests with:

     $ test/extended/core.sh

-This suite comprises many smaller suites, which are found under `test/extended` and can be run individually by

-specifying `--ginkgo.focus` and a regex filter:

+This suite comprises many smaller suites, which are found under `test/extended`

+and can be run individually by specifying `--ginkgo.focus` and a regex filter:

     $ test/extended/core.sh --ginkgo.focus=<regex>

-Extended tests should be Go tests in the `test/extended` directory that use the Ginkgo library. They

-must be able to be run remotely, and cannot depend on any local interaction with the filesystem or

-Docker.

+Extended tests should be Go tests in the `test/extended` directory that use

+the Ginkgo library. They must be able to be run remotely, and cannot depend on

+any local interaction with the filesystem or Docker.

 More information about running extended tests can be found in

-[test/extended/README](https://github.com/openshift/origin/blob/master/test/extended/README.md).

+[test/extended/README](https://github.com/openshift/origin/blob/master/test/exte

+nded/README.md).

 ## Installing Godep

-OpenShift and Kubernetes use [Godep](https://github.com/tools/godep) for dependency management.

-Godep allows versions of dependent packages to be locked at a specific commit by *vendoring* them

-(checking a copy of them into `vendor/`).  This means that everything you need for

-OpenShift is checked into this repository.  To install `godep` locally run:

+OpenShift and Kubernetes use [Godep](https://github.com/tools/godep) for

+dependency management.  Godep allows versions of dependent packages to be

+locked at a specific commit by *vendoring* them (checking a copy of them into

+`vendor/`).

+This means that everything you need for OpenShift is checked into this

+repository.

+

+To install `godep` locally run:

     $ go get github.com/tools/godep

@@ -229,53 +263,67 @@ If you are not updating packages you should not need godep installed.

 ## Cherry-picking an upstream commit into Origin: Why, how, and when.

-Origin carries patches inside of vendor/ on top of each rebase.  Thus, origin carries upstream patches in two ways.

+Origin carries patches inside of vendor/ on top of each rebase.

+Thus, origin carries upstream patches in two ways.

 1. *periodic rebases* against a Kubernetes commit.

-Eventually, any code you have in upstream kubernetes will land in Openshift via this mechanism.

+Eventually, any code you have in upstream kubernetes will land in Openshift

+via this mechanism.

-2. Cherry-picked patches for important *bug fixes*.  We really try to limit feature back-porting entirely.

+2. Cherry-picked patches for important *bug fixes*.  We really try to

+limit feature back-porting entirely.

 ### Manually

-You can manually try to cherry pick a commit (by using git apply). This can easily be done in a couple of steps.

+You can manually try to cherry pick a commit (by using git apply). This can

+easily be done in a couple of steps.

-- wget the patch, i.e. `wget -O /tmp/mypatch https://github.com/kubernetes/kubernetes/pull/34624.patch`

+- wget the patch, i.e. `wget -O /tmp/mypatch

+https://github.com/kubernetes/kubernetes/pull/34624.patch`

 - PATCH=/tmp/mypatch git apply --directory vendor/k8s.io/kubernetes $PATCH

 If this fails, then it's possible you may need to pick multiple commits.

-### For Openshift newcomers: Pick my kubernetes fix into Openshift vs. wait for the next rebase?

+### For Openshift newcomers: Pick my kubernetes fix into Openshift vs. wait for

+the next rebase?

-Assuming you read the bullets above... If your patch is really far behind, for example, if there have been 5 commits

-modifying the directory you care about, cherry picking will be increasingly difficult and you should consider waiting

-for the next rebase, which will likely include the commit you care about, or at least decrease the amount of cherry picks

-you need to do to merge.

+Assuming you read the bullets above... If your patch is really far behind, for

+example, if there have been 5 commits modifying the directory you care about, 

+cherry picking will be increasingly difficult and you should consider waiting

+for the next rebase, which will likely include the commit you care about, or at

+least decrease the amount of cherry picks you need to do to merge.

-To really know the answer, you need to know *how many commits behind you are in a particular directory*, often.

+To really know the answer, you need to know *how many commits behind you are in

+a particular directory*, often.

 To do this, just use git log, like so (using pkg/scheduler/ as an example).

 ```

-MYDIR=pkg/scheduler/algorithm git log --oneline -- vendor/k8s.io/kubernetes/${MYDIR} | grep UPSTREAM | cut -d' ' -f 4-10 | head -1

+MYDIR=pkg/scheduler/algorithm git log --oneline --

+vendor/k8s.io/kubernetes/${MYDIR} | grep UPSTREAM | cut -d' ' -f 4-10 | head -1

 ```

 The commit message printed above will tell you:

-- what the LAST commit in Kubernetes was (which effected "/pkg/scheduler/algorithm")

-- directory, which will give you an intuition about how "hot" the code you are cherry picking is.

-If it has changed a lot, recently, then

-that means you probably will want to wait for a rebase to land.

+

+- what the LAST commit in Kubernetes was (which effected

+"/pkg/scheduler/algorithm")

+- directory, which will give you an intuition about how "hot" the code you are

+cherry picking is.  If it has changed a lot, recently, then that means you

+probably will want to wait for a rebase to land.

 ### Using hack/cherry-pick

-For convenience, you can use `hack/cherry-pick.sh` to generate patches for Origin from upstream commits.

+For convenience, you can use `hack/cherry-pick.sh` to generate patches for

+Origin from upstream commits.

-The purpose of this command is to allow you to pull individual commits from a local kubernetes repository

-into origin's vendored kuberenetes in a fully automated manner.

+The purpose of this command is to allow you to pull individual commits from a

+local kubernetes repository into origin's vendored kuberenetes in a fully

+automated manner.

-To use this command, be sure to setup remote Pull Request branches in the kubernetes repository you are using

-(i.e. like https://gist.github.com/piscisaureus/3342247).  Specifically, you will be doing this, to the git config

-you probably already have for kubernetes:

+To use this command, be sure to setup remote Pull Request branches in the

+kubernetes repository you are using (i.e. like https://gist.github.com/piscisaureus/3342247).

+Specifically, you will be doing this, to the git config you probably already 

+have for kubernetes:

 ```

 [remote "origin"]

@@ -285,8 +333,11 @@ you probably already have for kubernetes:

         fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

 ```

-so that `git show origin/pr/<number>` displays information about your branch after a `git fetch`.

-You must also have the Kubernetes repository checked out in your GOPATH (visible as `../../../k8s.io/kubernetes`),

+so that `git show origin/pr/<number>` displays information about your branch

+after a `git fetch`.

+

+You must also have the Kubernetes repository checked out in your GOPATH

+(visible as `../../../k8s.io/kubernetes`),

 with openshift/kubernetes as a remote and fetched:

     $ pushd $GOPATH/src/k8s.io/kubernetes

@@ -300,13 +351,14 @@ To pull an upstream commit, run:

     $ hack/cherry-pick.sh <pr_number>

-This will attempt to create a patch from the current Kube rebase version in Origin that contains

-the commits added in the PR. If the PR has already been merged to the Kube version, you'll get an

-error. If there are conflicts, you'll have to resolve them in the upstream repo, then hit ENTER

-to continue. The end result will be a single commit in your Origin repo that contains the changes.

+This will attempt to create a patch from the current Kube rebase version in

+Origin that contains the commits added in the PR. If the PR has already been

+merged to the Kube version, you'll get an error. If there are conflicts, you'll

+have to resolve them in the upstream repo, then hit ENTER to continue. The end

+result will be a single commit in your Origin repo that contains the changes.

-If you want to run without a rebase option, set `NO_REBASE=1` before the command is run. You can

-also specify a commit range directly with:

+If you want to run without a rebase option, set `NO_REBASE=1` before the

+command is run. You can also specify a commit range directly with:

     $ hack/cherry-pick.sh origin/master...<some_branch>

@@ -314,29 +366,35 @@ All upstream commits should have a commit message where the first line is:

     UPSTREAM: <PR number|drop|carry>: <short description>

-`drop` indicates the commit should be removed during the next rebase. `carry` means that the change

-cannot go into upstream, and we should continue to use it during the next rebase.

+`drop` indicates the commit should be removed during the next rebase. `carry`

+means that the change cannot go into upstream, and we should continue to use it

+during the next rebase.

-You can also target repositories other than Kube by setting `UPSTREAM_REPO` and `UPSTREAM_PACKAGE`

-env vars.  `UPSTREAM_REPO` should be the full name of the Git repo as Go sees it, i.e.

-`github.com/coreos/etcd`, and `UPSTREAM_PACKAGE` must be a package inside that repo that is

-currently part of the Godeps.json file.  Example:

+You can also target repositories other than Kube by setting `UPSTREAM_REPO` and

+`UPSTREAM_PACKAGE` env vars.  `UPSTREAM_REPO` should be the full name of the Git

+repo as Go sees it, i.e. `github.com/coreos/etcd`, and `UPSTREAM_PACKAGE` must be

+a package inside that repo that is currently part of the Godeps.json file.  Example:

-    $ UPSTREAM_REPO=github.com/coreos/etcd UPSTREAM_PACKAGE=store hack/cherry-pick.sh <pr_number>

+    $ UPSTREAM_REPO=github.com/coreos/etcd UPSTREAM_PACKAGE=store

+hack/cherry-pick.sh <pr_number>

-By default `hack/cherry-pick.sh` uses git remote named `origin` to fetch kubernetes repository,

-if your git configuration is different, you can pass the git remote name by setting `UPSTREAM_REMOTE` env var:

+By default `hack/cherry-pick.sh` uses git remote named `origin` to fetch

+kubernetes repository, if your git configuration is different, you can pass the git

+remote name by setting `UPSTREAM_REMOTE` env var:

     $ UPSTREAM_REMOTE=upstream hack/cherry-pick.sh <pr_number>

 ## Moving a commit you developed in Origin to an upstream

-The `hack/move-upstream.sh` script takes the current feature branch, finds any changes to the

-requested upstream project (as defined by `UPSTREAM_REPO` and `UPSTREAM_PACKAGE`) that differ

-from `origin/master`, and then creates a new commit in that upstream project on a branch with

-the same name as your current branch.

+The `hack/move-upstream.sh` script takes the current feature branch, finds any

+changes to the

+requested upstream project (as defined by `UPSTREAM_REPO` and

+`UPSTREAM_PACKAGE`) that differ from `origin/master`, and then creates a new

+commit in that upstream project on a branch with the same name as your current

+branch.

-For example, to upstream a commit to OpenShift source-to-image while working from Origin:

+For example, to upstream a commit to OpenShift source-to-image while working

+from Origin:

     $ git checkout my_feature_branch_in_origin

     $ git log --oneline

@@ -345,10 +403,12 @@ For example, to upstream a commit to OpenShift source-to-image while working fro

     86eefdd UPSTREAM: 14618: Refactor exec to allow reuse from server

     # we want to move our STI changes to upstream

-    $ UPSTREAM_REPO=github.com/openshift/source-to-image UPSTREAM_PACKAGE=pkg/api hack/move-upstream.sh

+    $ UPSTREAM_REPO=github.com/openshift/source-to-image

+UPSTREAM_PACKAGE=pkg/api hack/move-upstream.sh

     ...

-    # All changes to source-to-image in Godeps/. are now in a commit UPSTREAMED in s2i repo

+    # All changes to source-to-image in Godeps/. are now in a commit UPSTREAMED

+in s2i repo

     $ cd ../source-to-image

     $ git log --oneline

@@ -356,19 +416,24 @@ For example, to upstream a commit to OpenShift source-to-image while working fro

     ... # older commits

 The default is to work against Kube.

-

+go

 ## Updating Kubernetes from upstream

-There are a few steps involved in rebasing Origin to a new version of Kubernetes. We need to make sure

-that not only the Kubernetes packages were updated correctly into `Godeps`, but also that *all tests are

-still running without errors* and *code changes, refactorings or the inclusion/removal of attributes

+There are a few steps involved in rebasing Origin to a new version of

+Kubernetes. We need to make sure

+that not only the Kubernetes packages were updated correctly into `Godeps`, but

+also that *all tests are

+still running without errors* and *code changes, refactorings or the

+inclusion/removal of attributes

 were properly reflected* in the Origin codebase.

 ### 1. Preparation

-Before you begin, make sure you have both [openshift/origin](https://github.com/openshift/origin) and

-[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) in your $GOPATH. You may want to work

+Before you begin, make sure you have both

+[openshift/origin](https://github.com/openshift/origin) and

+[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) in your

+$GOPATH. You may want to work

 on a separate $GOPATH just for the rebase:

 ```

@@ -384,7 +449,8 @@ $ git remote add openshift git@github.com:openshift/kubernetes.git

 $ git fetch openshift

 ```

-Check out the version of Kubernetes you want to rebase as a branch or tag named `stable_proposed` in

+Check out the version of Kubernetes you want to rebase as a branch or tag named

+`stable_proposed` in

 [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes). For example,

 if you are going to rebase the latest `master` of Kubernetes:

@@ -399,93 +465,126 @@ $ git checkout -b stable_proposed

 #### 2.1. First option (preferred): using the rebase-kube.sh script

-If all requirements described in *Preparation* were correctly attended, you should not have any trouble

-with rebasing the Kubernetes code using the script that automates this process.

+If all requirements described in *Preparation* were correctly attended, you

+should not have any trouble with rebasing the Kubernetes code using the script

+that automates this process.

 ```

 $ cd $GOPATH/src/github.com/openshift/origin

 $ hack/rebase-kube.sh

 ```

-Read over the changes with `git status` and make sure it looks reasonable. Check specially the

-`Godeps/Godeps.json` file to make sure no dependency is unintentionally missing.

+Read over the changes with `git status` and make sure it looks reasonable.

+Check specially the `Godeps/Godeps.json` file to make sure no dependency

+is unintentionally missing.

 Commit using the message `bump(k8s.io/kubernetes):<commit SHA>`, where

-`<commit SHA>` is the commit id for the Kubernetes version we are including in our Godeps. It can be

-found in our `Godeps/Godeps.json` in the declaration of any Kubernetes package.

+`<commit SHA>` is the commit id for the Kubernetes version we are including in

+our Godeps. It can be found in our `Godeps/Godeps.json` in the declaration of

+any Kubernetes package.

 #### 2.2. Second option: manually

-If for any reason you had trouble rebasing using the script, you may need to to do it manually.

-After following all requirements described in the *Preparation* topic, you will need to run

-`godep restore` from both the Origin and the Kubernetes directories and then `godep save ./...`

+If for any reason you had trouble rebasing using the script, you may need to to

+do it manually.

+After following all requirements described in the *Preparation* topic, you will

+need to run

+`godep restore` from both the Origin and the Kubernetes directories and then

+`godep save ./...`

 from the Origin directory. Follow these steps:

 1. `$ cd $GOPATH/src/github.com/openshift/origin`

-2. `make clean ; godep restore` will restore the package versions specified in the `Godeps/Godeps.json`

-of Origin to your GOPATH.

+2. `make clean ; godep restore` will restore the package versions specified in

+the `Godeps/Godeps.json` of Origin to your GOPATH.

 2. `$ cd $GOPATH/src/k8s.io/kubernetes`

-3. `$ git checkout stable_proposed` will checkout the desired version of Kubernetes as branched in

-*Preparation*.

-4. `$ godep restore` will restore the package versions specified in the `Godeps/Godeps.json`

-of Kubernetes to your GOPATH.

+3. `$ git checkout stable_proposed` will checkout the desired version of

+Kubernetes as branched in *Preparation*.

+4. `$ godep restore` will restore the package versions specified in the

+`Godeps/Godeps.json` of Kubernetes to your GOPATH.

 5. `$ cd $GOPATH/src/github.com/openshift/origin`.

-6. `$ make clean ; godep save ./...` will save a list of the checked-out dependencies to the file

-`Godeps/Godeps.json`, and copy their source code into `vendor`.

-7. If in the previous step godep complaints about the checked out revision of a package being different

-than the wanted revision, this probably means there are new packages in Kubernetes that we need to add.

-Do a `godep save <pkgname>` with the package specified by the error message and then `$ godep save ./...`

+6. `$ make clean ; godep save ./...` will save a list of the checked-out

+dependencies to the file `Godeps/Godeps.json`, and copy their source code into `vendor`.

+7. If in the previous step godep complaints about the checked out revision of a

+package being different  than the wanted revision, this probably means there are

+new packages in Kubernetes that we need to add.  Do a `godep save <pkgname>` with

+the package specified by the error message and then `$ godep save ./...`

 again.

-8. Read over the changes with `git status` and make sure it looks reasonable. Check specially the

-`Godeps/Godeps.json` file to make sure no dependency is unintentionally missing. The whole Godeps

-directory will be added to version control, including `_workspace`.

+8. Read over the changes with `git status` and make sure it looks reasonable.

+Check specially the `Godeps/Godeps.json` file to make sure no dependency is

+unintentionally missing. The whole Godeps directory will be added to version control, including

+`_workspace`.

 9. Commit using the message `bump(k8s.io/kubernetes):<commit SHA>`, where

-`<commit SHA>` is the commit id for the Kubernetes version we are including in our Godeps. It can be

-found in our `Godeps/Godeps.json` in the declaration of any Kubernetes package.

+`<commit SHA>` is the commit id for the Kubernetes version we are including in

+our Godeps. It can be found in our `Godeps/Godeps.json` in the declaration of

+any Kubernetes package.

-If in the process of rebasing manually you found any corner case not attended by the `hack/rebase-kube.sh`

-script, make sure you update it accordingly to help future rebases.

+If in the process of rebasing manually you found any corner case not attended

+by the `hack/rebase-kube.sh` script, make sure you update it accordingly to help future rebases.

 ### 3. cherry-pick upstream changes pushed to the Origin repo

-Eventually during the development cycle we introduce changes to dependencies right in the Origin

-repository. This is not a largely recommended practice, but it's useful if we need something that,

-for example, is in the Kubernetes repository but we are not doing a rebase yet. So, when doing the next

-rebase, we need to make sure we get all these changes otherwise they will be overridden by `godep save`.

-

-1. Check the `Godeps` directory [commits history](https://github.com/openshift/origin/commits/master/Godeps)

-for commits tagged with the *UPSTREAM* keyword. We will need to cherry-pick *all UPSTREAM commits since

-the last Kubernetes rebase* (remember you can find the last rebase commit looking for a message like

+Eventually during the development cycle we introduce changes to dependencies

+right in the Origin

+repository. This is not a largely recommended practice, but it's useful if we

+need something that,

+for example, is in the Kubernetes repository but we are not doing a rebase yet.

+So, when doing the next

+rebase, we need to make sure we get all these changes otherwise they will be

+overridden by `godep save`.

+

+1. Check the `Godeps` directory [commits

+history](https://github.com/openshift/origin/commits/master/Godeps)

+for commits tagged with the *UPSTREAM* keyword. We will need to cherry-pick

+*all UPSTREAM commits since

+the last Kubernetes rebase* (remember you can find the last rebase commit

+looking for a message like

 `bump(k8s.io/kubernetes):...`).

 2. For every commit tagged UPSTREAM, do `git cherry-pick <commit SHA>`.

-3. Notice that eventually the cherry-pick will be empty. This probably means the given change were

-already merged in Kubernetes and we don't need to specifically add it to our Godeps. Nice!

-4. Read over the commit history and make sure you have every UPSTREAM commit since the last rebase

+3. Notice that eventually the cherry-pick will be empty. This probably means

+the given change were

+already merged in Kubernetes and we don't need to specifically add it to our

+Godeps. Nice!

+4. Read over the commit history and make sure you have every UPSTREAM commit

+since the last rebase

 (except only for the empty ones).

 ### 4. Refactor Origin to be compliant with upstream changes

-After making sure we have all the dependencies in place and up-to-date, we need to work in the Origin

-codebase to make sure the compilation is not broken, all tests pass and it's compliant with any

-refactorings, architectural changes or behavior changes introduced in Kubernetes. Make sure:

+After making sure we have all the dependencies in place and up-to-date, we need

+to work in the Origin

+codebase to make sure the compilation is not broken, all tests pass and it's

+compliant with any refactorings, architectural changes or behavior changes

+introduced in Kubernetes. Make sure:

-1. `make clean ; hack/build-go.sh` compiles without errors and the standalone server starts correctly.

-1. all of our generated code is up to date by running all `hack/update-*` scripts.

+1. `make clean ; hack/build-go.sh` compiles without errors and the standalone

+server starts correctly.

+1. all of our generated code is up to date by running all `hack/update-*`

+scripts.

 1. `hack/verify-open-ports.sh` runs without errors.

-1. `hack/copy-kube-artifacts.sh` so Kubernetes tests can be fully functional. The diff resulting from this script should be squashed into the Kube bump commit.

+1. `hack/copy-kube-artifacts.sh` so Kubernetes tests can be fully functional.

+The diff resulting from this script should be squashed into the Kube bump

+commit.

 2. `TEST_KUBE=1 hack/test-go.sh` runs without errors.

 3. `hack/test-cmd.sh` runs without errors.

 3. `hack/test-integration.sh` runs without errors.

 3. `hack/test-end-to-end.sh` runs without errors.

-    See *Building a Release* above for setting up the environment for the *test-end-to-end.sh* tests.

+    See *Building a Release* above for setting up the environment for the

+*test-end-to-end.sh* tests.

-It is helpful to look at the Kubernetes commit history to be aware of the major topics. Although it

-can potentially break or change any part of Origin, the most affected parts are usually:

+It is helpful to look at the Kubernetes commit history to be aware of the major

+topics. Although it

+can potentially break or change any part of Origin, the most affected parts are

+usually:

 1. https://github.com/openshift/origin/blob/master/pkg/cmd/server/start

-2. https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go

-3. https://github.com/openshift/origin/blob/master/pkg/cmd/server/origin/master.go

-4. https://github.com/openshift/origin/blob/master/pkg/cmd/util/clientcmd/factory.go

+2.

+https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master

+.go

+3.

+https://github.com/openshift/origin/blob/master/pkg/cmd/server/origin/master.go

+4.

+https://github.com/openshift/origin/blob/master/pkg/cmd/util/clientcmd/factory.g

+o

 5. https://github.com/openshift/origin/blob/master/pkg/cmd/cli/cli.go

 6. https://github.com/openshift/origin/blob/master/pkg/api/meta/meta.go

@@ -495,28 +594,38 @@ Place all your changes in a commit called "Refactor to match changes upstream".

 A typical pull request for your Kubernetes rebase will contain:

-1. One commit for the Kuberentes Godeps bump (`bump(k8s.io/kubernetes):<commit SHA>`).

-2. Zero, one, or more bump commits for any **shared** dependencies between Origin and Kubernetes that have been bumped. Any transitive dependencies coming from Kubernetes should be squashed in the Kube bump commit.

+1. One commit for the Kuberentes Godeps bump (`bump(k8s.io/kubernetes):<commit

+SHA>`).

+2. Zero, one, or more bump commits for any **shared** dependencies between

+Origin and Kubernetes that have been bumped. Any transitive dependencies coming

+from Kubernetes should be squashed in the Kube bump commit.

 3. Zero, one, or more cherry-picked commits tagged UPSTREAM.

-4. One commit "Boring refactor to match changes upstream" that includes boring changes like imports rewriting, etc.

-5. One commit "Interesting refactor to match changes upstream" that includes interesting changes like new plugins or controller changes.

+4. One commit "Boring refactor to match changes upstream" that includes boring

+changes like imports rewriting, etc.

+5. One commit "Interesting refactor to match changes upstream" that includes

+interesting changes like new plugins or controller changes.

 ## Updating other Godeps from upstream

-To update to a new version of a dependency that's not already included in Kubernetes, checkout the

-correct version in your GOPATH and then run `godep save <pkgname>`.  This should create a new

-version of `Godeps/Godeps.json`, and update `vendor`.  Create a commit that includes

-both of these changes with message `bump(<pkgname>): <pkgcommit>`.

+To update to a new version of a dependency that's not already included in

+Kubernetes, checkout the correct version in your GOPATH and then run

+`godep save <pkgname>`.  This should create a new version of `Godeps/Godeps.json`,

+and update `vendor`.  Create a commit that includes both of these changes with message

+`bump(<pkgname>): <pkgcommit>`.

 ## Updating external examples

-`hack/update-external-example.sh` will pull down example files from external repositories and deposit them under the `examples` directory.

-Run this script if you need to refresh an example file, or add a new one.  See the script and `examples/quickstarts/README.md` for more details.

+`hack/update-external-example.sh` will pull down example files from external

+repositories and deposit them under the `examples` directory.

+Run this script if you need to refresh an example file, or add a new one.  See

+the script and `examples/quickstarts/README.md` for more details.

 ## Troubleshooting

 If you run into difficulties running OpenShift, start by reading through the

-[troubleshooting guide](https://github.com/openshift/origin/blob/master/docs/debugging-openshift.md).

+[troubleshooting

+guide](https://github.com/openshift/origin/blob/master/docs/debugging-openshift.

+md).

 ## RPM Packaging

@@ -529,8 +638,10 @@ and tag releases.

 ## GSSAPI-enabled builds

-When built with GSSAPI support, the `oc` client supports logging in with Kerberos credentials on Linux and OS X.

-GSSAPI-enabled builds of `oc` cannot be cross-compiled, but must be built on the target platform with the GSSAPI header files available.

+When built with GSSAPI support, the `oc` client supports logging in with

+Kerberos credentials on Linux and OS X.

+GSSAPI-enabled builds of `oc` cannot be cross-compiled, but must be built on

+the target platform with the GSSAPI header files available.

 On Linux, ensure the `krb5-devel` package is installed:

@@ -544,7 +655,7 @@ Once dependencies are in place, build with the `gssapi` tag:

     $ hack/build-go.sh cmd/oc -tags=gssapi

-Verify that the GSSAPI feature is enabled with `oc version`: 

+Verify that the GSSAPI feature is enabled with `oc version`:

     $ oc version

     ...

@@ -552,34 +663,57 @@ Verify that the GSSAPI feature is enabled with `oc version`:

 ## Swagger API Documentation

-OpenShift and Kubernetes integrate with the [Swagger 2.0 API framework](http://swagger.io) which aims to make it easier to document and write clients for RESTful APIs.  When you start OpenShift, the Swagger API endpoint is exposed at `https://localhost:8443/swaggerapi`. The Swagger UI makes it easy to view your documentation - to view the docs for your local version of OpenShift start the server with CORS enabled:

+OpenShift and Kubernetes integrate with the [Swagger 2.0 API

+framework](http://swagger.io) which aims to make it easier to document and

+write clients for RESTful APIs.  When you start OpenShift, the Swagger API

+endpoint is exposed at `https://localhost:8443/swaggerapi`. The Swagger UI

+makes it easy to view your documentation - to view the docs for your local

+version of OpenShift start the server with CORS enabled:

     $ openshift start --cors-allowed-origins=.*

-and then browse to http://openshift3swagger-claytondev.rhcloud.com (which runs a copy of the Swagger UI that points to localhost:8080 by default).  Expand the operations available on v1 to see the schemas (and to try the API directly). Additionally, you can download swagger-ui from http://swagger.io/swagger-ui/ and use it to point to your local swagger API endpoint.

+and then browse to http://openshift3swagger-claytondev.rhcloud.com (which runs

+a copy of the Swagger UI that points to localhost:8080 by default).  Expand the

+operations available on v1 to see the schemas (and to try the API directly).

+Additionally, you can download swagger-ui from http://swagger.io/swagger-ui/

+and use it to point to your local swagger API endpoint.

-Note: Hosted API documentation can be found [here](http://docs.openshift.org/latest/rest_api/openshift_v1.html).

+Note: Hosted API documentation can be found

+[here](http://docs.openshift.org/latest/rest_api/openshift_v1.html).

 ## Performance debugging

-OpenShift integrates the go `pprof` tooling to make it easy to capture CPU and heap dumps for running systems.  The following modes are available for the `openshift` binary (including all the CLI variants):

+OpenShift integrates the go `pprof` tooling to make it easy to capture CPU and

+heap dumps for running systems.  The following modes are available for the

+`openshift` binary (including all the CLI variants):

 * `OPENSHIFT_PROFILE` environment variable:

-  * `cpu` - will start a CPU profile on startup and write `./cpu.pprof`.  Contains samples for the entire run at the native sampling resolution (100hz). Note: CPU profiling for Go does not currently work on Mac OS X - the stats are not correctly sampled

-  * `mem` - generate a running heap dump that tracks allocations to `./mem.pprof`

+  * `cpu` - will start a CPU profile on startup and write `./cpu.pprof`.

+Contains samples for the entire run at the native sampling resolution (100hz).

+Note: CPU profiling for Go does not currently work on Mac OS X - the stats are

+not correctly sampled

+  * `mem` - generate a running heap dump that tracks allocations to

+`./mem.pprof`

   * `block` -  will start a block wait time analysis and write `./block.pprof`

-  * `web` - start the pprof webserver in process at http://127.0.0.1:6060/debug/pprof (you can open this in a browser). This supports `OPENSHIFT_PROFILE_HOST=` and `OPENSHIFT_PROFILE_PORT=` to change default ip `127.0.0.1` and default port `6060`.

+  * `web` - start the pprof webserver in process at http://127.0.0.1:6060/debug/pprof

+(you can open this in a browser). This supports `OPENSHIFT_PROFILE_HOST=`

+and `OPENSHIFT_PROFILE_PORT=` to change default ip `127.0.0.1` and default port `6060`.

 In order to start the server in CPU profiling mode, run:

     $ OPENSHIFT_PROFILE=cpu sudo ./_output/local/bin/linux/amd64/openshift start

-Or, if running OpenShift under systemd, append this to `/etc/sysconfig/atomic-openshift-{master,node}`

+Or, if running OpenShift under systemd, append this to

+`/etc/sysconfig/atomic-openshift-{master,node}`

     OPENSHIFT_PROFILE=cpu

-To view profiles, you use [pprof](http://goog-perftools.sourceforge.net/doc/cpu_profiler.html) which is part of `go tool`.  You must pass the binary you are debugging (for symbols) and a captured pprof.  For instance, to view a `cpu` profile from above, you would run OpenShift to completion, and then run:

+To view profiles, you use

+[pprof](http://goog-perftools.sourceforge.net/doc/cpu_profiler.html) which is

+part of `go tool`.  You must pass the binary you are debugging (for symbols)

+and a captured pprof.  For instance, to view a `cpu` profile from above, you

+would run OpenShift to completion, and then run:

     $ go tool pprof ./_output/local/bin/linux/amd64/openshift cpu.pprof

     or

@@ -603,20 +737,28 @@ to see the top20 CPU consuming fields or

 to launch a web browser window showing you where CPU time is going.

-`pprof` supports CLI arguments for looking at profiles in different ways - memory profiles by default show allocated space:

+`pprof` supports CLI arguments for looking at profiles in different ways -

+memory profiles by default show allocated space:

     $ go tool pprof ./_output/local/bin/linux/amd64/openshift mem.pprof

 but you can also see the allocated object counts:

-    $ go tool pprof --alloc_objects ./_output/local/bin/linux/amd64/openshift mem.pprof

+    $ go tool pprof --alloc_objects ./_output/local/bin/linux/amd64/openshift

+mem.pprof

-Finally, when using the `web` profile mode, you can have the go tool directly fetch your profiles via HTTP:

+Finally, when using the `web` profile mode, you can have the go tool directly

+fetch your profiles via HTTP:

     # for a 30s CPU trace

-    $ go tool pprof ./_output/local/bin/linux/amd64/openshift http://127.0.0.1:6060/debug/pprof/profile

+    $ go tool pprof ./_output/local/bin/linux/amd64/openshift

+http://127.0.0.1:6060/debug/pprof/profile

     # for a snapshot heap dump at the current time, showing total allocations

-    $ go tool pprof --alloc_space ./_output/local/bin/linux/amd64/openshift http://127.0.0.1:6060/debug/pprof/heap

+    $ go tool pprof --alloc_space ./_output/local/bin/linux/amd64/openshift

+http://127.0.0.1:6060/debug/pprof/heap

-See [debugging Go programs](https://golang.org/pkg/net/http/pprof/) for more info.  `pprof` has many modes and is very powerful (try `tree`) - you can pass a regex to many arguments to limit your results to only those samples that match the regex (basically the function name or the call stack).

+See [debugging Go programs](https://golang.org/pkg/net/http/pprof/) for more

+info.  `pprof` has many modes and is very powerful (try `tree`) - you can pass

+a regex to many arguments to limit your results to only those samples that

+match the regex (basically the function name or the call stack).