Browse code

vendor github.com/moby/buildkit/session

Signed-off-by: Tibor Vass <tibor@docker.com>

Tibor Vass authored on 2017/07/29 08:01:19
Showing 13 changed files
... ...
@@ -3,6 +3,7 @@ github.com/Azure/go-ansiterm 388960b655244e76e24c75f48631564eaefade62
3 3
 github.com/Microsoft/hcsshim v0.5.25
4 4
 github.com/Microsoft/go-winio v0.4.2
5 5
 github.com/Sirupsen/logrus v0.11.0
6
+github.com/moby/buildkit fed5c1d9cee6f734f58f3addca6e8d1750df48a6
6 7
 github.com/davecgh/go-spew 346938d642f2ec3594ed81d874461961cd0faa76
7 8
 github.com/docker/libtrust 9cbd2a1374f46905c68a4eb3694a130610adc62a
8 9
 github.com/go-check/check 4ed411733c5785b40214c70bce814c3a3a689609 https://github.com/cpuguy83/check.git
... ...
@@ -143,4 +144,4 @@ github.com/opencontainers/selinux v1.0.0-rc1
143 143
 # git --git-dir ./go/.git --work-tree ./go checkout revert-prefix-ignore
144 144
 # cp -a go/src/archive/tar ./vendor/archive/tar
145 145
 # rm -rf ./go
146
-# vndr
147 146
\ No newline at end of file
147
+# vndr
148 148
new file mode 100644
... ...
@@ -0,0 +1,201 @@
0
+                                 Apache License
1
+                           Version 2.0, January 2004
2
+                        http://www.apache.org/licenses/
3
+
4
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
5
+
6
+   1. Definitions.
7
+
8
+      "License" shall mean the terms and conditions for use, reproduction,
9
+      and distribution as defined by Sections 1 through 9 of this document.
10
+
11
+      "Licensor" shall mean the copyright owner or entity authorized by
12
+      the copyright owner that is granting the License.
13
+
14
+      "Legal Entity" shall mean the union of the acting entity and all
15
+      other entities that control, are controlled by, or are under common
16
+      control with that entity. For the purposes of this definition,
17
+      "control" means (i) the power, direct or indirect, to cause the
18
+      direction or management of such entity, whether by contract or
19
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
20
+      outstanding shares, or (iii) beneficial ownership of such entity.
21
+
22
+      "You" (or "Your") shall mean an individual or Legal Entity
23
+      exercising permissions granted by this License.
24
+
25
+      "Source" form shall mean the preferred form for making modifications,
26
+      including but not limited to software source code, documentation
27
+      source, and configuration files.
28
+
29
+      "Object" form shall mean any form resulting from mechanical
30
+      transformation or translation of a Source form, including but
31
+      not limited to compiled object code, generated documentation,
32
+      and conversions to other media types.
33
+
34
+      "Work" shall mean the work of authorship, whether in Source or
35
+      Object form, made available under the License, as indicated by a
36
+      copyright notice that is included in or attached to the work
37
+      (an example is provided in the Appendix below).
38
+
39
+      "Derivative Works" shall mean any work, whether in Source or Object
40
+      form, that is based on (or derived from) the Work and for which the
41
+      editorial revisions, annotations, elaborations, or other modifications
42
+      represent, as a whole, an original work of authorship. For the purposes
43
+      of this License, Derivative Works shall not include works that remain
44
+      separable from, or merely link (or bind by name) to the interfaces of,
45
+      the Work and Derivative Works thereof.
46
+
47
+      "Contribution" shall mean any work of authorship, including
48
+      the original version of the Work and any modifications or additions
49
+      to that Work or Derivative Works thereof, that is intentionally
50
+      submitted to Licensor for inclusion in the Work by the copyright owner
51
+      or by an individual or Legal Entity authorized to submit on behalf of
52
+      the copyright owner. For the purposes of this definition, "submitted"
53
+      means any form of electronic, verbal, or written communication sent
54
+      to the Licensor or its representatives, including but not limited to
55
+      communication on electronic mailing lists, source code control systems,
56
+      and issue tracking systems that are managed by, or on behalf of, the
57
+      Licensor for the purpose of discussing and improving the Work, but
58
+      excluding communication that is conspicuously marked or otherwise
59
+      designated in writing by the copyright owner as "Not a Contribution."
60
+
61
+      "Contributor" shall mean Licensor and any individual or Legal Entity
62
+      on behalf of whom a Contribution has been received by Licensor and
63
+      subsequently incorporated within the Work.
64
+
65
+   2. Grant of Copyright License. Subject to the terms and conditions of
66
+      this License, each Contributor hereby grants to You a perpetual,
67
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
68
+      copyright license to reproduce, prepare Derivative Works of,
69
+      publicly display, publicly perform, sublicense, and distribute the
70
+      Work and such Derivative Works in Source or Object form.
71
+
72
+   3. Grant of Patent License. Subject to the terms and conditions of
73
+      this License, each Contributor hereby grants to You a perpetual,
74
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
75
+      (except as stated in this section) patent license to make, have made,
76
+      use, offer to sell, sell, import, and otherwise transfer the Work,
77
+      where such license applies only to those patent claims licensable
78
+      by such Contributor that are necessarily infringed by their
79
+      Contribution(s) alone or by combination of their Contribution(s)
80
+      with the Work to which such Contribution(s) was submitted. If You
81
+      institute patent litigation against any entity (including a
82
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
83
+      or a Contribution incorporated within the Work constitutes direct
84
+      or contributory patent infringement, then any patent licenses
85
+      granted to You under this License for that Work shall terminate
86
+      as of the date such litigation is filed.
87
+
88
+   4. Redistribution. You may reproduce and distribute copies of the
89
+      Work or Derivative Works thereof in any medium, with or without
90
+      modifications, and in Source or Object form, provided that You
91
+      meet the following conditions:
92
+
93
+      (a) You must give any other recipients of the Work or
94
+          Derivative Works a copy of this License; and
95
+
96
+      (b) You must cause any modified files to carry prominent notices
97
+          stating that You changed the files; and
98
+
99
+      (c) You must retain, in the Source form of any Derivative Works
100
+          that You distribute, all copyright, patent, trademark, and
101
+          attribution notices from the Source form of the Work,
102
+          excluding those notices that do not pertain to any part of
103
+          the Derivative Works; and
104
+
105
+      (d) If the Work includes a "NOTICE" text file as part of its
106
+          distribution, then any Derivative Works that You distribute must
107
+          include a readable copy of the attribution notices contained
108
+          within such NOTICE file, excluding those notices that do not
109
+          pertain to any part of the Derivative Works, in at least one
110
+          of the following places: within a NOTICE text file distributed
111
+          as part of the Derivative Works; within the Source form or
112
+          documentation, if provided along with the Derivative Works; or,
113
+          within a display generated by the Derivative Works, if and
114
+          wherever such third-party notices normally appear. The contents
115
+          of the NOTICE file are for informational purposes only and
116
+          do not modify the License. You may add Your own attribution
117
+          notices within Derivative Works that You distribute, alongside
118
+          or as an addendum to the NOTICE text from the Work, provided
119
+          that such additional attribution notices cannot be construed
120
+          as modifying the License.
121
+
122
+      You may add Your own copyright statement to Your modifications and
123
+      may provide additional or different license terms and conditions
124
+      for use, reproduction, or distribution of Your modifications, or
125
+      for any such Derivative Works as a whole, provided Your use,
126
+      reproduction, and distribution of the Work otherwise complies with
127
+      the conditions stated in this License.
128
+
129
+   5. Submission of Contributions. Unless You explicitly state otherwise,
130
+      any Contribution intentionally submitted for inclusion in the Work
131
+      by You to the Licensor shall be under the terms and conditions of
132
+      this License, without any additional terms or conditions.
133
+      Notwithstanding the above, nothing herein shall supersede or modify
134
+      the terms of any separate license agreement you may have executed
135
+      with Licensor regarding such Contributions.
136
+
137
+   6. Trademarks. This License does not grant permission to use the trade
138
+      names, trademarks, service marks, or product names of the Licensor,
139
+      except as required for reasonable and customary use in describing the
140
+      origin of the Work and reproducing the content of the NOTICE file.
141
+
142
+   7. Disclaimer of Warranty. Unless required by applicable law or
143
+      agreed to in writing, Licensor provides the Work (and each
144
+      Contributor provides its Contributions) on an "AS IS" BASIS,
145
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
146
+      implied, including, without limitation, any warranties or conditions
147
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
148
+      PARTICULAR PURPOSE. You are solely responsible for determining the
149
+      appropriateness of using or redistributing the Work and assume any
150
+      risks associated with Your exercise of permissions under this License.
151
+
152
+   8. Limitation of Liability. In no event and under no legal theory,
153
+      whether in tort (including negligence), contract, or otherwise,
154
+      unless required by applicable law (such as deliberate and grossly
155
+      negligent acts) or agreed to in writing, shall any Contributor be
156
+      liable to You for damages, including any direct, indirect, special,
157
+      incidental, or consequential damages of any character arising as a
158
+      result of this License or out of the use or inability to use the
159
+      Work (including but not limited to damages for loss of goodwill,
160
+      work stoppage, computer failure or malfunction, or any and all
161
+      other commercial damages or losses), even if such Contributor
162
+      has been advised of the possibility of such damages.
163
+
164
+   9. Accepting Warranty or Additional Liability. While redistributing
165
+      the Work or Derivative Works thereof, You may choose to offer,
166
+      and charge a fee for, acceptance of support, warranty, indemnity,
167
+      or other liability obligations and/or rights consistent with this
168
+      License. However, in accepting such obligations, You may act only
169
+      on Your own behalf and on Your sole responsibility, not on behalf
170
+      of any other Contributor, and only if You agree to indemnify,
171
+      defend, and hold each Contributor harmless for any liability
172
+      incurred by, or claims asserted against, such Contributor by reason
173
+      of your accepting any such warranty or additional liability.
174
+
175
+   END OF TERMS AND CONDITIONS
176
+
177
+   APPENDIX: How to apply the Apache License to your work.
178
+
179
+      To apply the Apache License to your work, attach the following
180
+      boilerplate notice, with the fields enclosed by brackets "[]"
181
+      replaced with your own identifying information. (Don't include
182
+      the brackets!)  The text should be enclosed in the appropriate
183
+      comment syntax for the file format. We also recommend that a
184
+      file or class name and description of purpose be included on the
185
+      same "printed page" as the copyright notice for easier
186
+      identification within third-party archives.
187
+
188
+   Copyright [yyyy] [name of copyright owner]
189
+
190
+   Licensed under the Apache License, Version 2.0 (the "License");
191
+   you may not use this file except in compliance with the License.
192
+   You may obtain a copy of the License at
193
+
194
+       http://www.apache.org/licenses/LICENSE-2.0
195
+
196
+   Unless required by applicable law or agreed to in writing, software
197
+   distributed under the License is distributed on an "AS IS" BASIS,
198
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
199
+   See the License for the specific language governing permissions and
200
+   limitations under the License.
0 201
new file mode 100644
... ...
@@ -0,0 +1,87 @@
0
+### Important: This repository is in an early development phase and not suitable for practical workloads. It does not compare with `docker build` features yet.
1
+
2
+[![asciicinema example](https://asciinema.org/a/gPEIEo1NzmDTUu2bEPsUboqmU.png)](https://asciinema.org/a/gPEIEo1NzmDTUu2bEPsUboqmU)
3
+
4
+
5
+## BuildKit
6
+
7
+BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner.
8
+
9
+Key features:
10
+- Automatic garbage collection
11
+- Extendable frontend formats
12
+- Concurrent dependency resolution
13
+- Efficient instruction caching
14
+- Build cache import/export
15
+- Nested build job invocations
16
+- Distributable workers
17
+- Multiple output formats
18
+- Pluggable architecture
19
+
20
+
21
+Read the proposal from https://github.com/moby/moby/issues/32925
22
+
23
+#### Quick start
24
+
25
+BuildKit daemon can be built in two different versions: one that uses [containerd](https://github.com/containerd/containerd) for execution and distribution, and a standalone version that doesn't have other dependencies apart from [runc](https://github.com/opencontainers/runc). We are open for adding more backends. `buildd` is a CLI utility for running the gRPC API. 
26
+
27
+```bash
28
+# buildd daemon (choose one)
29
+go build -o buildd-containerd -tags containerd ./cmd/buildd
30
+go build -o buildd-standalone -tags standalone ./cmd/buildd
31
+
32
+# buildctl utility
33
+go build -o buildctl ./cmd/buildctl
34
+```
35
+
36
+You can also use `make binaries` that prepares all binaries into the `bin/` directory.
37
+
38
+The first thing to test could be to try building BuildKit with BuildKit. BuildKit provides a low-level solver format that could be used by multiple build definitions. Preparation work for making the Dockerfile parser reusable as a frontend is tracked in https://github.com/moby/moby/pull/33492. As no frontends have been integrated yet we currently have to use a client library to generate this low-level definition.
39
+
40
+`examples/buildkit*` directory contains scripts that define how to build different configurations of BuildKit and its dependencies using the `client` package. Running one of these script generates a protobuf definition of a build graph. Note that the script itself does not execute any steps of the build.
41
+
42
+You can use `buildctl debug dump-llb` to see what data is this definition.
43
+
44
+```bash
45
+go run examples/buildkit0/buildkit.go | buildctl debug dump-llb | jq .
46
+```
47
+
48
+To start building use `buildctl build` command. The script accepts `--target` flag to choose between `containerd` and `standalone` configurations. In standalone mode BuildKit binaries are built together with `runc`. In containerd mode, the `containerd` binary is built as well from the upstream repo.
49
+
50
+```bash
51
+go run examples/buildkit0/buildkit.go | buildctl build
52
+```
53
+
54
+`buildctl build` will show interactive progress bar by default while the build job is running. It will also show you the path to the trace file that contains all information about the timing of the individual steps and logs.
55
+
56
+Different versions of the example scripts show different ways of describing the build definition for this project to show the capabilities of the library. New versions have been added when new features have become available.
57
+
58
+- `./examples/buildkit0` - uses only exec operations, defines a full stage per component.
59
+- `./examples/buildkit1` - cloning git repositories has been separated for extra concurrency.
60
+- `./examples/buildkit2` - uses git sources directly instead of running `git clone`, allowing better performance and much safer caching.
61
+
62
+#### Supported runc version
63
+
64
+During development buildkit is tested with the version of runc that is being used by the containerd repository. Please refer to [runc.md](https://github.com/containerd/containerd/blob/3707703a694187c7d08e2f333da6ddd58bcb729d/RUNC.md) for more information.
65
+
66
+
67
+#### Contributing
68
+
69
+Running tests:
70
+
71
+```bash
72
+make test
73
+```
74
+
75
+Updating vendored dependencies:
76
+
77
+```bash
78
+# update vendor.conf
79
+make vendor
80
+```
81
+
82
+Validating your updates before submission:
83
+
84
+```bash
85
+make validate-all
86
+```
0 87
new file mode 100644
... ...
@@ -0,0 +1,31 @@
0
+package filesync
1
+
2
+import (
3
+	"time"
4
+
5
+	"google.golang.org/grpc"
6
+
7
+	"github.com/Sirupsen/logrus"
8
+	"github.com/tonistiigi/fsutil"
9
+)
10
+
11
+func sendDiffCopy(stream grpc.Stream, dir string, includes, excludes []string, progress progressCb) error {
12
+	return fsutil.Send(stream.Context(), stream, dir, &fsutil.WalkOpt{
13
+		ExcludePatterns: excludes,
14
+		IncludePaths:    includes, // TODO: rename IncludePatterns
15
+	}, progress)
16
+}
17
+
18
+func recvDiffCopy(ds grpc.Stream, dest string, cu CacheUpdater) error {
19
+	st := time.Now()
20
+	defer func() {
21
+		logrus.Debugf("diffcopy took: %v", time.Since(st))
22
+	}()
23
+	var cf fsutil.ChangeFunc
24
+	if cu != nil {
25
+		cu.MarkSupported(true)
26
+		cf = cu.HandleChange
27
+	}
28
+
29
+	return fsutil.Receive(ds.Context(), ds, dest, cf)
30
+}
0 31
new file mode 100644
... ...
@@ -0,0 +1,183 @@
0
+package filesync
1
+
2
+import (
3
+	"os"
4
+	"strings"
5
+
6
+	"github.com/moby/buildkit/session"
7
+	"github.com/pkg/errors"
8
+	"github.com/tonistiigi/fsutil"
9
+	"golang.org/x/net/context"
10
+	"google.golang.org/grpc"
11
+	"google.golang.org/grpc/metadata"
12
+)
13
+
14
+const (
15
+	keyOverrideExcludes = "override-excludes"
16
+	keyIncludePatterns  = "include-patterns"
17
+)
18
+
19
+type fsSyncProvider struct {
20
+	root     string
21
+	excludes []string
22
+	p        progressCb
23
+	doneCh   chan error
24
+}
25
+
26
+// NewFSSyncProvider creates a new provider for sending files from client
27
+func NewFSSyncProvider(root string, excludes []string) session.Attachable {
28
+	p := &fsSyncProvider{
29
+		root:     root,
30
+		excludes: excludes,
31
+	}
32
+	return p
33
+}
34
+
35
+func (sp *fsSyncProvider) Register(server *grpc.Server) {
36
+	RegisterFileSyncServer(server, sp)
37
+}
38
+
39
+func (sp *fsSyncProvider) DiffCopy(stream FileSync_DiffCopyServer) error {
40
+	return sp.handle("diffcopy", stream)
41
+}
42
+func (sp *fsSyncProvider) TarStream(stream FileSync_TarStreamServer) error {
43
+	return sp.handle("tarstream", stream)
44
+}
45
+
46
+func (sp *fsSyncProvider) handle(method string, stream grpc.ServerStream) error {
47
+	var pr *protocol
48
+	for _, p := range supportedProtocols {
49
+		if method == p.name && isProtoSupported(p.name) {
50
+			pr = &p
51
+			break
52
+		}
53
+	}
54
+	if pr == nil {
55
+		return errors.New("failed to negotiate protocol")
56
+	}
57
+
58
+	opts, _ := metadata.FromContext(stream.Context()) // if no metadata continue with empty object
59
+
60
+	var excludes []string
61
+	if len(opts[keyOverrideExcludes]) == 0 || opts[keyOverrideExcludes][0] != "true" {
62
+		excludes = sp.excludes
63
+	}
64
+	includes := opts[keyIncludePatterns]
65
+
66
+	var progress progressCb
67
+	if sp.p != nil {
68
+		progress = sp.p
69
+		sp.p = nil
70
+	}
71
+
72
+	var doneCh chan error
73
+	if sp.doneCh != nil {
74
+		doneCh = sp.doneCh
75
+		sp.doneCh = nil
76
+	}
77
+	err := pr.sendFn(stream, sp.root, includes, excludes, progress)
78
+	if doneCh != nil {
79
+		if err != nil {
80
+			doneCh <- err
81
+		}
82
+		close(doneCh)
83
+	}
84
+	return err
85
+}
86
+
87
+func (sp *fsSyncProvider) SetNextProgressCallback(f func(int, bool), doneCh chan error) {
88
+	sp.p = f
89
+	sp.doneCh = doneCh
90
+}
91
+
92
+type progressCb func(int, bool)
93
+
94
+type protocol struct {
95
+	name   string
96
+	sendFn func(stream grpc.Stream, srcDir string, includes, excludes []string, progress progressCb) error
97
+	recvFn func(stream grpc.Stream, destDir string, cu CacheUpdater) error
98
+}
99
+
100
+func isProtoSupported(p string) bool {
101
+	// TODO: this should be removed after testing if stability is confirmed
102
+	if override := os.Getenv("BUILD_STREAM_PROTOCOL"); override != "" {
103
+		return strings.EqualFold(p, override)
104
+	}
105
+	return true
106
+}
107
+
108
+var supportedProtocols = []protocol{
109
+	{
110
+		name:   "diffcopy",
111
+		sendFn: sendDiffCopy,
112
+		recvFn: recvDiffCopy,
113
+	},
114
+	{
115
+		name:   "tarstream",
116
+		sendFn: sendTarStream,
117
+		recvFn: recvTarStream,
118
+	},
119
+}
120
+
121
+// FSSendRequestOpt defines options for FSSend request
122
+type FSSendRequestOpt struct {
123
+	IncludePatterns  []string
124
+	OverrideExcludes bool
125
+	DestDir          string
126
+	CacheUpdater     CacheUpdater
127
+}
128
+
129
+// CacheUpdater is an object capable of sending notifications for the cache hash changes
130
+type CacheUpdater interface {
131
+	MarkSupported(bool)
132
+	HandleChange(fsutil.ChangeKind, string, os.FileInfo, error) error
133
+}
134
+
135
+// FSSync initializes a transfer of files
136
+func FSSync(ctx context.Context, c session.Caller, opt FSSendRequestOpt) error {
137
+	var pr *protocol
138
+	for _, p := range supportedProtocols {
139
+		if isProtoSupported(p.name) && c.Supports(session.MethodURL(_FileSync_serviceDesc.ServiceName, p.name)) {
140
+			pr = &p
141
+			break
142
+		}
143
+	}
144
+	if pr == nil {
145
+		return errors.New("no fssync handlers")
146
+	}
147
+
148
+	opts := make(map[string][]string)
149
+	if opt.OverrideExcludes {
150
+		opts[keyOverrideExcludes] = []string{"true"}
151
+	}
152
+
153
+	if opt.IncludePatterns != nil {
154
+		opts[keyIncludePatterns] = opt.IncludePatterns
155
+	}
156
+
157
+	ctx, cancel := context.WithCancel(ctx)
158
+	defer cancel()
159
+
160
+	client := NewFileSyncClient(c.Conn())
161
+
162
+	var stream grpc.ClientStream
163
+
164
+	ctx = metadata.NewContext(ctx, opts)
165
+
166
+	switch pr.name {
167
+	case "tarstream":
168
+		cc, err := client.TarStream(ctx)
169
+		if err != nil {
170
+			return err
171
+		}
172
+		stream = cc
173
+	case "diffcopy":
174
+		cc, err := client.DiffCopy(ctx)
175
+		if err != nil {
176
+			return err
177
+		}
178
+		stream = cc
179
+	}
180
+
181
+	return pr.recvFn(stream, opt.DestDir, opt.CacheUpdater)
182
+}
0 183
new file mode 100644
... ...
@@ -0,0 +1,575 @@
0
+// Code generated by protoc-gen-gogo.
1
+// source: filesync.proto
2
+// DO NOT EDIT!
3
+
4
+/*
5
+Package filesync is a generated protocol buffer package.
6
+
7
+It is generated from these files:
8
+	filesync.proto
9
+
10
+It has these top-level messages:
11
+	BytesMessage
12
+*/
13
+package filesync
14
+
15
+import proto "github.com/gogo/protobuf/proto"
16
+import fmt "fmt"
17
+import math "math"
18
+
19
+import bytes "bytes"
20
+
21
+import strings "strings"
22
+import reflect "reflect"
23
+
24
+import (
25
+	context "golang.org/x/net/context"
26
+	grpc "google.golang.org/grpc"
27
+)
28
+
29
+import io "io"
30
+
31
+// Reference imports to suppress errors if they are not otherwise used.
32
+var _ = proto.Marshal
33
+var _ = fmt.Errorf
34
+var _ = math.Inf
35
+
36
+// This is a compile-time assertion to ensure that this generated file
37
+// is compatible with the proto package it is being compiled against.
38
+// A compilation error at this line likely means your copy of the
39
+// proto package needs to be updated.
40
+const _ = proto.GoGoProtoPackageIsVersion2 // please upgrade the proto package
41
+
42
+// BytesMessage contains a chunk of byte data
43
+type BytesMessage struct {
44
+	Data []byte `protobuf:"bytes,1,opt,name=data,proto3" json:"data,omitempty"`
45
+}
46
+
47
+func (m *BytesMessage) Reset()                    { *m = BytesMessage{} }
48
+func (*BytesMessage) ProtoMessage()               {}
49
+func (*BytesMessage) Descriptor() ([]byte, []int) { return fileDescriptorFilesync, []int{0} }
50
+
51
+func (m *BytesMessage) GetData() []byte {
52
+	if m != nil {
53
+		return m.Data
54
+	}
55
+	return nil
56
+}
57
+
58
+func init() {
59
+	proto.RegisterType((*BytesMessage)(nil), "moby.filesync.v1.BytesMessage")
60
+}
61
+func (this *BytesMessage) Equal(that interface{}) bool {
62
+	if that == nil {
63
+		if this == nil {
64
+			return true
65
+		}
66
+		return false
67
+	}
68
+
69
+	that1, ok := that.(*BytesMessage)
70
+	if !ok {
71
+		that2, ok := that.(BytesMessage)
72
+		if ok {
73
+			that1 = &that2
74
+		} else {
75
+			return false
76
+		}
77
+	}
78
+	if that1 == nil {
79
+		if this == nil {
80
+			return true
81
+		}
82
+		return false
83
+	} else if this == nil {
84
+		return false
85
+	}
86
+	if !bytes.Equal(this.Data, that1.Data) {
87
+		return false
88
+	}
89
+	return true
90
+}
91
+func (this *BytesMessage) GoString() string {
92
+	if this == nil {
93
+		return "nil"
94
+	}
95
+	s := make([]string, 0, 5)
96
+	s = append(s, "&filesync.BytesMessage{")
97
+	s = append(s, "Data: "+fmt.Sprintf("%#v", this.Data)+",\n")
98
+	s = append(s, "}")
99
+	return strings.Join(s, "")
100
+}
101
+func valueToGoStringFilesync(v interface{}, typ string) string {
102
+	rv := reflect.ValueOf(v)
103
+	if rv.IsNil() {
104
+		return "nil"
105
+	}
106
+	pv := reflect.Indirect(rv).Interface()
107
+	return fmt.Sprintf("func(v %v) *%v { return &v } ( %#v )", typ, typ, pv)
108
+}
109
+
110
+// Reference imports to suppress errors if they are not otherwise used.
111
+var _ context.Context
112
+var _ grpc.ClientConn
113
+
114
+// This is a compile-time assertion to ensure that this generated file
115
+// is compatible with the grpc package it is being compiled against.
116
+const _ = grpc.SupportPackageIsVersion4
117
+
118
+// Client API for FileSync service
119
+
120
+type FileSyncClient interface {
121
+	DiffCopy(ctx context.Context, opts ...grpc.CallOption) (FileSync_DiffCopyClient, error)
122
+	TarStream(ctx context.Context, opts ...grpc.CallOption) (FileSync_TarStreamClient, error)
123
+}
124
+
125
+type fileSyncClient struct {
126
+	cc *grpc.ClientConn
127
+}
128
+
129
+func NewFileSyncClient(cc *grpc.ClientConn) FileSyncClient {
130
+	return &fileSyncClient{cc}
131
+}
132
+
133
+func (c *fileSyncClient) DiffCopy(ctx context.Context, opts ...grpc.CallOption) (FileSync_DiffCopyClient, error) {
134
+	stream, err := grpc.NewClientStream(ctx, &_FileSync_serviceDesc.Streams[0], c.cc, "/moby.filesync.v1.FileSync/DiffCopy", opts...)
135
+	if err != nil {
136
+		return nil, err
137
+	}
138
+	x := &fileSyncDiffCopyClient{stream}
139
+	return x, nil
140
+}
141
+
142
+type FileSync_DiffCopyClient interface {
143
+	Send(*BytesMessage) error
144
+	Recv() (*BytesMessage, error)
145
+	grpc.ClientStream
146
+}
147
+
148
+type fileSyncDiffCopyClient struct {
149
+	grpc.ClientStream
150
+}
151
+
152
+func (x *fileSyncDiffCopyClient) Send(m *BytesMessage) error {
153
+	return x.ClientStream.SendMsg(m)
154
+}
155
+
156
+func (x *fileSyncDiffCopyClient) Recv() (*BytesMessage, error) {
157
+	m := new(BytesMessage)
158
+	if err := x.ClientStream.RecvMsg(m); err != nil {
159
+		return nil, err
160
+	}
161
+	return m, nil
162
+}
163
+
164
+func (c *fileSyncClient) TarStream(ctx context.Context, opts ...grpc.CallOption) (FileSync_TarStreamClient, error) {
165
+	stream, err := grpc.NewClientStream(ctx, &_FileSync_serviceDesc.Streams[1], c.cc, "/moby.filesync.v1.FileSync/TarStream", opts...)
166
+	if err != nil {
167
+		return nil, err
168
+	}
169
+	x := &fileSyncTarStreamClient{stream}
170
+	return x, nil
171
+}
172
+
173
+type FileSync_TarStreamClient interface {
174
+	Send(*BytesMessage) error
175
+	Recv() (*BytesMessage, error)
176
+	grpc.ClientStream
177
+}
178
+
179
+type fileSyncTarStreamClient struct {
180
+	grpc.ClientStream
181
+}
182
+
183
+func (x *fileSyncTarStreamClient) Send(m *BytesMessage) error {
184
+	return x.ClientStream.SendMsg(m)
185
+}
186
+
187
+func (x *fileSyncTarStreamClient) Recv() (*BytesMessage, error) {
188
+	m := new(BytesMessage)
189
+	if err := x.ClientStream.RecvMsg(m); err != nil {
190
+		return nil, err
191
+	}
192
+	return m, nil
193
+}
194
+
195
+// Server API for FileSync service
196
+
197
+type FileSyncServer interface {
198
+	DiffCopy(FileSync_DiffCopyServer) error
199
+	TarStream(FileSync_TarStreamServer) error
200
+}
201
+
202
+func RegisterFileSyncServer(s *grpc.Server, srv FileSyncServer) {
203
+	s.RegisterService(&_FileSync_serviceDesc, srv)
204
+}
205
+
206
+func _FileSync_DiffCopy_Handler(srv interface{}, stream grpc.ServerStream) error {
207
+	return srv.(FileSyncServer).DiffCopy(&fileSyncDiffCopyServer{stream})
208
+}
209
+
210
+type FileSync_DiffCopyServer interface {
211
+	Send(*BytesMessage) error
212
+	Recv() (*BytesMessage, error)
213
+	grpc.ServerStream
214
+}
215
+
216
+type fileSyncDiffCopyServer struct {
217
+	grpc.ServerStream
218
+}
219
+
220
+func (x *fileSyncDiffCopyServer) Send(m *BytesMessage) error {
221
+	return x.ServerStream.SendMsg(m)
222
+}
223
+
224
+func (x *fileSyncDiffCopyServer) Recv() (*BytesMessage, error) {
225
+	m := new(BytesMessage)
226
+	if err := x.ServerStream.RecvMsg(m); err != nil {
227
+		return nil, err
228
+	}
229
+	return m, nil
230
+}
231
+
232
+func _FileSync_TarStream_Handler(srv interface{}, stream grpc.ServerStream) error {
233
+	return srv.(FileSyncServer).TarStream(&fileSyncTarStreamServer{stream})
234
+}
235
+
236
+type FileSync_TarStreamServer interface {
237
+	Send(*BytesMessage) error
238
+	Recv() (*BytesMessage, error)
239
+	grpc.ServerStream
240
+}
241
+
242
+type fileSyncTarStreamServer struct {
243
+	grpc.ServerStream
244
+}
245
+
246
+func (x *fileSyncTarStreamServer) Send(m *BytesMessage) error {
247
+	return x.ServerStream.SendMsg(m)
248
+}
249
+
250
+func (x *fileSyncTarStreamServer) Recv() (*BytesMessage, error) {
251
+	m := new(BytesMessage)
252
+	if err := x.ServerStream.RecvMsg(m); err != nil {
253
+		return nil, err
254
+	}
255
+	return m, nil
256
+}
257
+
258
+var _FileSync_serviceDesc = grpc.ServiceDesc{
259
+	ServiceName: "moby.filesync.v1.FileSync",
260
+	HandlerType: (*FileSyncServer)(nil),
261
+	Methods:     []grpc.MethodDesc{},
262
+	Streams: []grpc.StreamDesc{
263
+		{
264
+			StreamName:    "DiffCopy",
265
+			Handler:       _FileSync_DiffCopy_Handler,
266
+			ServerStreams: true,
267
+			ClientStreams: true,
268
+		},
269
+		{
270
+			StreamName:    "TarStream",
271
+			Handler:       _FileSync_TarStream_Handler,
272
+			ServerStreams: true,
273
+			ClientStreams: true,
274
+		},
275
+	},
276
+	Metadata: "filesync.proto",
277
+}
278
+
279
+func (m *BytesMessage) Marshal() (dAtA []byte, err error) {
280
+	size := m.Size()
281
+	dAtA = make([]byte, size)
282
+	n, err := m.MarshalTo(dAtA)
283
+	if err != nil {
284
+		return nil, err
285
+	}
286
+	return dAtA[:n], nil
287
+}
288
+
289
+func (m *BytesMessage) MarshalTo(dAtA []byte) (int, error) {
290
+	var i int
291
+	_ = i
292
+	var l int
293
+	_ = l
294
+	if len(m.Data) > 0 {
295
+		dAtA[i] = 0xa
296
+		i++
297
+		i = encodeVarintFilesync(dAtA, i, uint64(len(m.Data)))
298
+		i += copy(dAtA[i:], m.Data)
299
+	}
300
+	return i, nil
301
+}
302
+
303
+func encodeFixed64Filesync(dAtA []byte, offset int, v uint64) int {
304
+	dAtA[offset] = uint8(v)
305
+	dAtA[offset+1] = uint8(v >> 8)
306
+	dAtA[offset+2] = uint8(v >> 16)
307
+	dAtA[offset+3] = uint8(v >> 24)
308
+	dAtA[offset+4] = uint8(v >> 32)
309
+	dAtA[offset+5] = uint8(v >> 40)
310
+	dAtA[offset+6] = uint8(v >> 48)
311
+	dAtA[offset+7] = uint8(v >> 56)
312
+	return offset + 8
313
+}
314
+func encodeFixed32Filesync(dAtA []byte, offset int, v uint32) int {
315
+	dAtA[offset] = uint8(v)
316
+	dAtA[offset+1] = uint8(v >> 8)
317
+	dAtA[offset+2] = uint8(v >> 16)
318
+	dAtA[offset+3] = uint8(v >> 24)
319
+	return offset + 4
320
+}
321
+func encodeVarintFilesync(dAtA []byte, offset int, v uint64) int {
322
+	for v >= 1<<7 {
323
+		dAtA[offset] = uint8(v&0x7f | 0x80)
324
+		v >>= 7
325
+		offset++
326
+	}
327
+	dAtA[offset] = uint8(v)
328
+	return offset + 1
329
+}
330
+func (m *BytesMessage) Size() (n int) {
331
+	var l int
332
+	_ = l
333
+	l = len(m.Data)
334
+	if l > 0 {
335
+		n += 1 + l + sovFilesync(uint64(l))
336
+	}
337
+	return n
338
+}
339
+
340
+func sovFilesync(x uint64) (n int) {
341
+	for {
342
+		n++
343
+		x >>= 7
344
+		if x == 0 {
345
+			break
346
+		}
347
+	}
348
+	return n
349
+}
350
+func sozFilesync(x uint64) (n int) {
351
+	return sovFilesync(uint64((x << 1) ^ uint64((int64(x) >> 63))))
352
+}
353
+func (this *BytesMessage) String() string {
354
+	if this == nil {
355
+		return "nil"
356
+	}
357
+	s := strings.Join([]string{`&BytesMessage{`,
358
+		`Data:` + fmt.Sprintf("%v", this.Data) + `,`,
359
+		`}`,
360
+	}, "")
361
+	return s
362
+}
363
+func valueToStringFilesync(v interface{}) string {
364
+	rv := reflect.ValueOf(v)
365
+	if rv.IsNil() {
366
+		return "nil"
367
+	}
368
+	pv := reflect.Indirect(rv).Interface()
369
+	return fmt.Sprintf("*%v", pv)
370
+}
371
+func (m *BytesMessage) Unmarshal(dAtA []byte) error {
372
+	l := len(dAtA)
373
+	iNdEx := 0
374
+	for iNdEx < l {
375
+		preIndex := iNdEx
376
+		var wire uint64
377
+		for shift := uint(0); ; shift += 7 {
378
+			if shift >= 64 {
379
+				return ErrIntOverflowFilesync
380
+			}
381
+			if iNdEx >= l {
382
+				return io.ErrUnexpectedEOF
383
+			}
384
+			b := dAtA[iNdEx]
385
+			iNdEx++
386
+			wire |= (uint64(b) & 0x7F) << shift
387
+			if b < 0x80 {
388
+				break
389
+			}
390
+		}
391
+		fieldNum := int32(wire >> 3)
392
+		wireType := int(wire & 0x7)
393
+		if wireType == 4 {
394
+			return fmt.Errorf("proto: BytesMessage: wiretype end group for non-group")
395
+		}
396
+		if fieldNum <= 0 {
397
+			return fmt.Errorf("proto: BytesMessage: illegal tag %d (wire type %d)", fieldNum, wire)
398
+		}
399
+		switch fieldNum {
400
+		case 1:
401
+			if wireType != 2 {
402
+				return fmt.Errorf("proto: wrong wireType = %d for field Data", wireType)
403
+			}
404
+			var byteLen int
405
+			for shift := uint(0); ; shift += 7 {
406
+				if shift >= 64 {
407
+					return ErrIntOverflowFilesync
408
+				}
409
+				if iNdEx >= l {
410
+					return io.ErrUnexpectedEOF
411
+				}
412
+				b := dAtA[iNdEx]
413
+				iNdEx++
414
+				byteLen |= (int(b) & 0x7F) << shift
415
+				if b < 0x80 {
416
+					break
417
+				}
418
+			}
419
+			if byteLen < 0 {
420
+				return ErrInvalidLengthFilesync
421
+			}
422
+			postIndex := iNdEx + byteLen
423
+			if postIndex > l {
424
+				return io.ErrUnexpectedEOF
425
+			}
426
+			m.Data = append(m.Data[:0], dAtA[iNdEx:postIndex]...)
427
+			if m.Data == nil {
428
+				m.Data = []byte{}
429
+			}
430
+			iNdEx = postIndex
431
+		default:
432
+			iNdEx = preIndex
433
+			skippy, err := skipFilesync(dAtA[iNdEx:])
434
+			if err != nil {
435
+				return err
436
+			}
437
+			if skippy < 0 {
438
+				return ErrInvalidLengthFilesync
439
+			}
440
+			if (iNdEx + skippy) > l {
441
+				return io.ErrUnexpectedEOF
442
+			}
443
+			iNdEx += skippy
444
+		}
445
+	}
446
+
447
+	if iNdEx > l {
448
+		return io.ErrUnexpectedEOF
449
+	}
450
+	return nil
451
+}
452
+func skipFilesync(dAtA []byte) (n int, err error) {
453
+	l := len(dAtA)
454
+	iNdEx := 0
455
+	for iNdEx < l {
456
+		var wire uint64
457
+		for shift := uint(0); ; shift += 7 {
458
+			if shift >= 64 {
459
+				return 0, ErrIntOverflowFilesync
460
+			}
461
+			if iNdEx >= l {
462
+				return 0, io.ErrUnexpectedEOF
463
+			}
464
+			b := dAtA[iNdEx]
465
+			iNdEx++
466
+			wire |= (uint64(b) & 0x7F) << shift
467
+			if b < 0x80 {
468
+				break
469
+			}
470
+		}
471
+		wireType := int(wire & 0x7)
472
+		switch wireType {
473
+		case 0:
474
+			for shift := uint(0); ; shift += 7 {
475
+				if shift >= 64 {
476
+					return 0, ErrIntOverflowFilesync
477
+				}
478
+				if iNdEx >= l {
479
+					return 0, io.ErrUnexpectedEOF
480
+				}
481
+				iNdEx++
482
+				if dAtA[iNdEx-1] < 0x80 {
483
+					break
484
+				}
485
+			}
486
+			return iNdEx, nil
487
+		case 1:
488
+			iNdEx += 8
489
+			return iNdEx, nil
490
+		case 2:
491
+			var length int
492
+			for shift := uint(0); ; shift += 7 {
493
+				if shift >= 64 {
494
+					return 0, ErrIntOverflowFilesync
495
+				}
496
+				if iNdEx >= l {
497
+					return 0, io.ErrUnexpectedEOF
498
+				}
499
+				b := dAtA[iNdEx]
500
+				iNdEx++
501
+				length |= (int(b) & 0x7F) << shift
502
+				if b < 0x80 {
503
+					break
504
+				}
505
+			}
506
+			iNdEx += length
507
+			if length < 0 {
508
+				return 0, ErrInvalidLengthFilesync
509
+			}
510
+			return iNdEx, nil
511
+		case 3:
512
+			for {
513
+				var innerWire uint64
514
+				var start int = iNdEx
515
+				for shift := uint(0); ; shift += 7 {
516
+					if shift >= 64 {
517
+						return 0, ErrIntOverflowFilesync
518
+					}
519
+					if iNdEx >= l {
520
+						return 0, io.ErrUnexpectedEOF
521
+					}
522
+					b := dAtA[iNdEx]
523
+					iNdEx++
524
+					innerWire |= (uint64(b) & 0x7F) << shift
525
+					if b < 0x80 {
526
+						break
527
+					}
528
+				}
529
+				innerWireType := int(innerWire & 0x7)
530
+				if innerWireType == 4 {
531
+					break
532
+				}
533
+				next, err := skipFilesync(dAtA[start:])
534
+				if err != nil {
535
+					return 0, err
536
+				}
537
+				iNdEx = start + next
538
+			}
539
+			return iNdEx, nil
540
+		case 4:
541
+			return iNdEx, nil
542
+		case 5:
543
+			iNdEx += 4
544
+			return iNdEx, nil
545
+		default:
546
+			return 0, fmt.Errorf("proto: illegal wireType %d", wireType)
547
+		}
548
+	}
549
+	panic("unreachable")
550
+}
551
+
552
+var (
553
+	ErrInvalidLengthFilesync = fmt.Errorf("proto: negative length found during unmarshaling")
554
+	ErrIntOverflowFilesync   = fmt.Errorf("proto: integer overflow")
555
+)
556
+
557
+func init() { proto.RegisterFile("filesync.proto", fileDescriptorFilesync) }
558
+
559
+var fileDescriptorFilesync = []byte{
560
+	// 198 bytes of a gzipped FileDescriptorProto
561
+	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0xe2, 0xe2, 0x4b, 0xcb, 0xcc, 0x49,
562
+	0x2d, 0xae, 0xcc, 0x4b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x12, 0xc8, 0xcd, 0x4f, 0xaa,
563
+	0xd4, 0x83, 0x0b, 0x96, 0x19, 0x2a, 0x29, 0x71, 0xf1, 0x38, 0x55, 0x96, 0xa4, 0x16, 0xfb, 0xa6,
564
+	0x16, 0x17, 0x27, 0xa6, 0xa7, 0x0a, 0x09, 0x71, 0xb1, 0xa4, 0x24, 0x96, 0x24, 0x4a, 0x30, 0x2a,
565
+	0x30, 0x6a, 0xf0, 0x04, 0x81, 0xd9, 0x46, 0xab, 0x19, 0xb9, 0x38, 0xdc, 0x32, 0x73, 0x52, 0x83,
566
+	0x2b, 0xf3, 0x92, 0x85, 0xfc, 0xb8, 0x38, 0x5c, 0x32, 0xd3, 0xd2, 0x9c, 0xf3, 0x0b, 0x2a, 0x85,
567
+	0xe4, 0xf4, 0xd0, 0xcd, 0xd3, 0x43, 0x36, 0x4c, 0x8a, 0x80, 0xbc, 0x06, 0xa3, 0x01, 0xa3, 0x90,
568
+	0x3f, 0x17, 0x67, 0x48, 0x62, 0x51, 0x70, 0x49, 0x51, 0x6a, 0x62, 0x2e, 0x35, 0x0c, 0x74, 0x32,
569
+	0xbb, 0xf0, 0x50, 0x8e, 0xe1, 0xc6, 0x43, 0x39, 0x86, 0x0f, 0x0f, 0xe5, 0x18, 0x1b, 0x1e, 0xc9,
570
+	0x31, 0xae, 0x78, 0x24, 0xc7, 0x78, 0xe2, 0x91, 0x1c, 0xe3, 0x85, 0x47, 0x72, 0x8c, 0x0f, 0x1e,
571
+	0xc9, 0x31, 0xbe, 0x78, 0x24, 0xc7, 0xf0, 0xe1, 0x91, 0x1c, 0xe3, 0x84, 0xc7, 0x72, 0x0c, 0x51,
572
+	0x1c, 0x30, 0xb3, 0x92, 0xd8, 0xc0, 0x41, 0x64, 0x0c, 0x08, 0x00, 0x00, 0xff, 0xff, 0x5f, 0x0c,
573
+	0x8d, 0xc5, 0x34, 0x01, 0x00, 0x00,
574
+}
0 575
new file mode 100644
... ...
@@ -0,0 +1,15 @@
0
+syntax = "proto3";
1
+
2
+package moby.filesync.v1;
3
+
4
+option go_package = "filesync";
5
+
6
+service FileSync{
7
+  rpc DiffCopy(stream BytesMessage) returns (stream BytesMessage);
8
+  rpc TarStream(stream BytesMessage) returns (stream BytesMessage);
9
+}
10
+
11
+// BytesMessage contains a chunk of byte data
12
+message BytesMessage{
13
+	bytes data = 1;
14
+}
0 15
\ No newline at end of file
1 16
new file mode 100644
... ...
@@ -0,0 +1,3 @@
0
+package filesync
1
+
2
+//go:generate protoc --gogoslick_out=plugins=grpc:. filesync.proto
0 3
new file mode 100644
... ...
@@ -0,0 +1,83 @@
0
+package filesync
1
+
2
+import (
3
+	"io"
4
+
5
+	"github.com/Sirupsen/logrus"
6
+	"github.com/docker/docker/pkg/archive"
7
+	"github.com/docker/docker/pkg/chrootarchive"
8
+	"github.com/pkg/errors"
9
+	"google.golang.org/grpc"
10
+)
11
+
12
+func sendTarStream(stream grpc.Stream, dir string, includes, excludes []string, progress progressCb) error {
13
+	a, err := archive.TarWithOptions(dir, &archive.TarOptions{
14
+		ExcludePatterns: excludes,
15
+	})
16
+	if err != nil {
17
+		return err
18
+	}
19
+
20
+	size := 0
21
+	buf := make([]byte, 1<<15)
22
+	t := new(BytesMessage)
23
+	for {
24
+		n, err := a.Read(buf)
25
+		if err != nil {
26
+			if err == io.EOF {
27
+				break
28
+			}
29
+			return err
30
+		}
31
+		t.Data = buf[:n]
32
+
33
+		if err := stream.SendMsg(t); err != nil {
34
+			return err
35
+		}
36
+		size += n
37
+		if progress != nil {
38
+			progress(size, false)
39
+		}
40
+	}
41
+	if progress != nil {
42
+		progress(size, true)
43
+	}
44
+	return nil
45
+}
46
+
47
+func recvTarStream(ds grpc.Stream, dest string, cs CacheUpdater) error {
48
+
49
+	pr, pw := io.Pipe()
50
+
51
+	go func() {
52
+		var (
53
+			err error
54
+			t   = new(BytesMessage)
55
+		)
56
+		for {
57
+			if err = ds.RecvMsg(t); err != nil {
58
+				if err == io.EOF {
59
+					err = nil
60
+				}
61
+				break
62
+			}
63
+			_, err = pw.Write(t.Data)
64
+			if err != nil {
65
+				break
66
+			}
67
+		}
68
+		if err = pw.CloseWithError(err); err != nil {
69
+			logrus.Errorf("failed to close tar transfer pipe")
70
+		}
71
+	}()
72
+
73
+	decompressedStream, err := archive.DecompressStream(pr)
74
+	if err != nil {
75
+		return errors.Wrap(err, "failed to decompress stream")
76
+	}
77
+
78
+	if err := chrootarchive.Untar(decompressedStream, dest, nil); err != nil {
79
+		return errors.Wrap(err, "failed to untar context")
80
+	}
81
+	return nil
82
+}
0 83
new file mode 100644
... ...
@@ -0,0 +1,62 @@
0
+package session
1
+
2
+import (
3
+	"net"
4
+	"time"
5
+
6
+	"github.com/Sirupsen/logrus"
7
+	"github.com/pkg/errors"
8
+	"golang.org/x/net/context"
9
+	"golang.org/x/net/http2"
10
+	"google.golang.org/grpc"
11
+	"google.golang.org/grpc/health/grpc_health_v1"
12
+)
13
+
14
+func serve(ctx context.Context, grpcServer *grpc.Server, conn net.Conn) {
15
+	go func() {
16
+		<-ctx.Done()
17
+		conn.Close()
18
+	}()
19
+	logrus.Debugf("serving grpc connection")
20
+	(&http2.Server{}).ServeConn(conn, &http2.ServeConnOpts{Handler: grpcServer})
21
+}
22
+
23
+func grpcClientConn(ctx context.Context, conn net.Conn) (context.Context, *grpc.ClientConn, error) {
24
+	dialOpt := grpc.WithDialer(func(addr string, d time.Duration) (net.Conn, error) {
25
+		return conn, nil
26
+	})
27
+
28
+	cc, err := grpc.DialContext(ctx, "", dialOpt, grpc.WithInsecure())
29
+	if err != nil {
30
+		return nil, nil, errors.Wrap(err, "failed to create grpc client")
31
+	}
32
+
33
+	ctx, cancel := context.WithCancel(ctx)
34
+	go monitorHealth(ctx, cc, cancel)
35
+
36
+	return ctx, cc, nil
37
+}
38
+
39
+func monitorHealth(ctx context.Context, cc *grpc.ClientConn, cancelConn func()) {
40
+	defer cancelConn()
41
+	defer cc.Close()
42
+
43
+	ticker := time.NewTicker(500 * time.Millisecond)
44
+	defer ticker.Stop()
45
+	healthClient := grpc_health_v1.NewHealthClient(cc)
46
+
47
+	for {
48
+		select {
49
+		case <-ctx.Done():
50
+			return
51
+		case <-ticker.C:
52
+			<-ticker.C
53
+			ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
54
+			_, err := healthClient.Check(ctx, &grpc_health_v1.HealthCheckRequest{})
55
+			cancel()
56
+			if err != nil {
57
+				return
58
+			}
59
+		}
60
+	}
61
+}
0 62
new file mode 100644
... ...
@@ -0,0 +1,202 @@
0
+package session
1
+
2
+import (
3
+	"net"
4
+	"net/http"
5
+	"strings"
6
+	"sync"
7
+
8
+	"github.com/pkg/errors"
9
+	"golang.org/x/net/context"
10
+	"google.golang.org/grpc"
11
+)
12
+
13
+// Caller can invoke requests on the session
14
+type Caller interface {
15
+	Context() context.Context
16
+	Supports(method string) bool
17
+	Conn() *grpc.ClientConn
18
+	Name() string
19
+	SharedKey() string
20
+}
21
+
22
+type client struct {
23
+	Session
24
+	cc        *grpc.ClientConn
25
+	supported map[string]struct{}
26
+}
27
+
28
+// Manager is a controller for accessing currently active sessions
29
+type Manager struct {
30
+	sessions        map[string]*client
31
+	mu              sync.Mutex
32
+	updateCondition *sync.Cond
33
+}
34
+
35
+// NewManager returns a new Manager
36
+func NewManager() (*Manager, error) {
37
+	sm := &Manager{
38
+		sessions: make(map[string]*client),
39
+	}
40
+	sm.updateCondition = sync.NewCond(&sm.mu)
41
+	return sm, nil
42
+}
43
+
44
+// HandleHTTPRequest handles an incoming HTTP request
45
+func (sm *Manager) HandleHTTPRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) error {
46
+	hijacker, ok := w.(http.Hijacker)
47
+	if !ok {
48
+		return errors.New("handler does not support hijack")
49
+	}
50
+
51
+	uuid := r.Header.Get(headerSessionUUID)
52
+
53
+	proto := r.Header.Get("Upgrade")
54
+
55
+	sm.mu.Lock()
56
+	if _, ok := sm.sessions[uuid]; ok {
57
+		sm.mu.Unlock()
58
+		return errors.Errorf("session %s already exists", uuid)
59
+	}
60
+
61
+	if proto == "" {
62
+		sm.mu.Unlock()
63
+		return errors.New("no upgrade proto in request")
64
+	}
65
+
66
+	if proto != "h2c" {
67
+		sm.mu.Unlock()
68
+		return errors.Errorf("protocol %s not supported", proto)
69
+	}
70
+
71
+	conn, _, err := hijacker.Hijack()
72
+	if err != nil {
73
+		sm.mu.Unlock()
74
+		return errors.Wrap(err, "failed to hijack connection")
75
+	}
76
+
77
+	resp := &http.Response{
78
+		StatusCode: http.StatusSwitchingProtocols,
79
+		ProtoMajor: 1,
80
+		ProtoMinor: 1,
81
+		Header:     http.Header{},
82
+	}
83
+	resp.Header.Set("Connection", "Upgrade")
84
+	resp.Header.Set("Upgrade", proto)
85
+
86
+	// set raw mode
87
+	conn.Write([]byte{})
88
+	resp.Write(conn)
89
+
90
+	return sm.handleConn(ctx, conn, r.Header)
91
+}
92
+
93
+// HandleConn handles an incoming raw connection
94
+func (sm *Manager) HandleConn(ctx context.Context, conn net.Conn, opts map[string][]string) error {
95
+	sm.mu.Lock()
96
+	return sm.handleConn(ctx, conn, opts)
97
+}
98
+
99
+// caller needs to take lock, this function will release it
100
+func (sm *Manager) handleConn(ctx context.Context, conn net.Conn, opts map[string][]string) error {
101
+	ctx, cancel := context.WithCancel(ctx)
102
+	defer cancel()
103
+
104
+	h := http.Header(opts)
105
+	uuid := h.Get(headerSessionUUID)
106
+	name := h.Get(headerSessionName)
107
+	sharedKey := h.Get(headerSessionSharedKey)
108
+
109
+	ctx, cc, err := grpcClientConn(ctx, conn)
110
+	if err != nil {
111
+		sm.mu.Unlock()
112
+		return err
113
+	}
114
+
115
+	c := &client{
116
+		Session: Session{
117
+			uuid:      uuid,
118
+			name:      name,
119
+			sharedKey: sharedKey,
120
+			ctx:       ctx,
121
+			cancelCtx: cancel,
122
+			done:      make(chan struct{}),
123
+		},
124
+		cc:        cc,
125
+		supported: make(map[string]struct{}),
126
+	}
127
+
128
+	for _, m := range opts[headerSessionMethod] {
129
+		c.supported[strings.ToLower(m)] = struct{}{}
130
+	}
131
+	sm.sessions[uuid] = c
132
+	sm.updateCondition.Broadcast()
133
+	sm.mu.Unlock()
134
+
135
+	defer func() {
136
+		sm.mu.Lock()
137
+		delete(sm.sessions, uuid)
138
+		sm.mu.Unlock()
139
+	}()
140
+
141
+	<-c.ctx.Done()
142
+	conn.Close()
143
+	close(c.done)
144
+
145
+	return nil
146
+}
147
+
148
+// Get returns a session by UUID
149
+func (sm *Manager) Get(ctx context.Context, uuid string) (Caller, error) {
150
+	ctx, cancel := context.WithCancel(ctx)
151
+	defer cancel()
152
+
153
+	go func() {
154
+		select {
155
+		case <-ctx.Done():
156
+			sm.updateCondition.Broadcast()
157
+		}
158
+	}()
159
+
160
+	var c *client
161
+
162
+	sm.mu.Lock()
163
+	for {
164
+		select {
165
+		case <-ctx.Done():
166
+			sm.mu.Unlock()
167
+			return nil, errors.Wrapf(ctx.Err(), "no active session for %s", uuid)
168
+		default:
169
+		}
170
+		var ok bool
171
+		c, ok = sm.sessions[uuid]
172
+		if !ok || c.closed() {
173
+			sm.updateCondition.Wait()
174
+			continue
175
+		}
176
+		sm.mu.Unlock()
177
+		break
178
+	}
179
+
180
+	return c, nil
181
+}
182
+
183
+func (c *client) Context() context.Context {
184
+	return c.context()
185
+}
186
+
187
+func (c *client) Name() string {
188
+	return c.name
189
+}
190
+
191
+func (c *client) SharedKey() string {
192
+	return c.sharedKey
193
+}
194
+
195
+func (c *client) Supports(url string) bool {
196
+	_, ok := c.supported[strings.ToLower(url)]
197
+	return ok
198
+}
199
+func (c *client) Conn() *grpc.ClientConn {
200
+	return c.cc
201
+}
0 202
new file mode 100644
... ...
@@ -0,0 +1,117 @@
0
+package session
1
+
2
+import (
3
+	"net"
4
+
5
+	"github.com/docker/docker/pkg/stringid"
6
+	"github.com/pkg/errors"
7
+	"golang.org/x/net/context"
8
+	"google.golang.org/grpc"
9
+	"google.golang.org/grpc/health"
10
+	"google.golang.org/grpc/health/grpc_health_v1"
11
+)
12
+
13
+const (
14
+	headerSessionUUID      = "X-Docker-Expose-Session-Uuid"
15
+	headerSessionName      = "X-Docker-Expose-Session-Name"
16
+	headerSessionSharedKey = "X-Docker-Expose-Session-Sharedkey"
17
+	headerSessionMethod    = "X-Docker-Expose-Session-Grpc-Method"
18
+)
19
+
20
+// Dialer returns a connection that can be used by the session
21
+type Dialer func(ctx context.Context, proto string, meta map[string][]string) (net.Conn, error)
22
+
23
+// Attachable defines a feature that can be expsed on a session
24
+type Attachable interface {
25
+	Register(*grpc.Server)
26
+}
27
+
28
+// Session is a long running connection between client and a daemon
29
+type Session struct {
30
+	uuid       string
31
+	name       string
32
+	sharedKey  string
33
+	ctx        context.Context
34
+	cancelCtx  func()
35
+	done       chan struct{}
36
+	grpcServer *grpc.Server
37
+}
38
+
39
+// NewSession returns a new long running session
40
+func NewSession(name, sharedKey string) (*Session, error) {
41
+	uuid := stringid.GenerateRandomID()
42
+	s := &Session{
43
+		uuid:       uuid,
44
+		name:       name,
45
+		sharedKey:  sharedKey,
46
+		grpcServer: grpc.NewServer(),
47
+	}
48
+
49
+	grpc_health_v1.RegisterHealthServer(s.grpcServer, health.NewServer())
50
+
51
+	return s, nil
52
+}
53
+
54
+// Allow enable a given service to be reachable through the grpc session
55
+func (s *Session) Allow(a Attachable) {
56
+	a.Register(s.grpcServer)
57
+}
58
+
59
+// UUID returns unique identifier for the session
60
+func (s *Session) UUID() string {
61
+	return s.uuid
62
+}
63
+
64
+// Run activates the session
65
+func (s *Session) Run(ctx context.Context, dialer Dialer) error {
66
+	ctx, cancel := context.WithCancel(ctx)
67
+	s.cancelCtx = cancel
68
+	s.done = make(chan struct{})
69
+
70
+	defer cancel()
71
+	defer close(s.done)
72
+
73
+	meta := make(map[string][]string)
74
+	meta[headerSessionUUID] = []string{s.uuid}
75
+	meta[headerSessionName] = []string{s.name}
76
+	meta[headerSessionSharedKey] = []string{s.sharedKey}
77
+
78
+	for name, svc := range s.grpcServer.GetServiceInfo() {
79
+		for _, method := range svc.Methods {
80
+			meta[headerSessionMethod] = append(meta[headerSessionMethod], MethodURL(name, method.Name))
81
+		}
82
+	}
83
+	conn, err := dialer(ctx, "h2c", meta)
84
+	if err != nil {
85
+		return errors.Wrap(err, "failed to dial gRPC")
86
+	}
87
+	serve(ctx, s.grpcServer, conn)
88
+	return nil
89
+}
90
+
91
+// Close closes the session
92
+func (s *Session) Close() error {
93
+	if s.cancelCtx != nil && s.done != nil {
94
+		s.cancelCtx()
95
+		<-s.done
96
+	}
97
+	return nil
98
+}
99
+
100
+func (s *Session) context() context.Context {
101
+	return s.ctx
102
+}
103
+
104
+func (s *Session) closed() bool {
105
+	select {
106
+	case <-s.context().Done():
107
+		return true
108
+	default:
109
+		return false
110
+	}
111
+}
112
+
113
+// MethodURL returns a gRPC method URL for service and method name
114
+func MethodURL(s, m string) string {
115
+	return "/" + s + "/" + m
116
+}
0 117
new file mode 100644
... ...
@@ -0,0 +1,40 @@
0
+github.com/boltdb/bolt e9cf4fae01b5a8ff89d0ec6b32f0d9c9f79aefdd
1
+github.com/pkg/errors c605e284fe17294bda444b34710735b29d1a9d90
2
+
3
+github.com/stretchr/testify v1.1.4
4
+github.com/davecgh/go-spew v1.1.0
5
+github.com/pmezard/go-difflib v1.0.0
6
+golang.org/x/sys 739734461d1c916b6c72a63d7efda2b27edb369f
7
+
8
+github.com/containerd/containerd 3707703a694187c7d08e2f333da6ddd58bcb729d
9
+golang.org/x/sync 450f422ab23cf9881c94e2db30cac0eb1b7cf80c
10
+github.com/Sirupsen/logrus v0.11.0
11
+google.golang.org/grpc v1.3.0
12
+github.com/opencontainers/go-digest 21dfd564fd89c944783d00d069f33e3e7123c448
13
+golang.org/x/net 1f9224279e98554b6a6432d4dd998a739f8b2b7c
14
+github.com/gogo/protobuf d2e1ade2d719b78fe5b061b4c18a9f7111b5bdc8
15
+github.com/golang/protobuf 5a0f697c9ed9d68fef0116532c6e05cfeae00e55
16
+github.com/containerd/continuity 86cec1535a968310e7532819f699ff2830ed7463
17
+github.com/opencontainers/image-spec v1.0.0-rc6
18
+github.com/opencontainers/runc 429a5387123625040bacfbb60d96b1cbd02293ab
19
+github.com/Microsoft/go-winio v0.4.1
20
+github.com/containerd/fifo 69b99525e472735860a5269b75af1970142b3062
21
+github.com/opencontainers/runtime-spec 198f23f827eea397d4331d7eb048d9d4c7ff7bee
22
+github.com/containerd/go-runc 2774a2ea124a5c2d0aba13b5c2dd8a5a9a48775d
23
+github.com/containerd/console 7fed77e673ca4abcd0cbd6d4d0e0e22137cbd778
24
+github.com/Azure/go-ansiterm fa152c58bc15761d0200cb75fe958b89a9d4888e
25
+google.golang.org/genproto d80a6e20e776b0b17a324d0ba1ab50a39c8e8944
26
+golang.org/x/text 19e51611da83d6be54ddafce4a4af510cb3e9ea4
27
+github.com/docker/go-events aa2e3b613fbbfdddbe055a7b9e3ce271cfd83eca
28
+
29
+github.com/urfave/cli d70f47eeca3afd795160003bc6e28b001d60c67c
30
+github.com/docker/go-units 0dadbb0345b35ec7ef35e228dabb8de89a65bf52
31
+github.com/google/shlex 6f45313302b9c56850fc17f99e40caebce98c716
32
+golang.org/x/time 8be79e1e0910c292df4e79c241bb7e8f7e725959
33
+
34
+github.com/BurntSushi/locker 392720b78f44e9d0249fcac6c43b111b47a370b8
35
+github.com/docker/docker 05c7c311390911daebcf5d9519dee813fc02a887
36
+github.com/pkg/profile 5b67d428864e92711fcbd2f8629456121a56d91f
37
+
38
+github.com/tonistiigi/fsutil 0ac4c11b053b9c5c7c47558f81f96c7100ce50fb
39
+github.com/stevvooe/continuity 86cec1535a968310e7532819f699ff2830ed7463