Browse code
Add docs about how to extend devicemapper thin pool
Signed-off-by: Chun Chen <ramichen@tencent.com>
Chun Chen authored on 2016/04/05 16:35:24Showing 2 changed files
| 1 | 1 |
new file mode 100644 |
| ... | ... |
@@ -0,0 +1,14 @@ |
| 0 |
+Docker device tool for devicemapper storage driver backend |
|
| 1 |
+=================== |
|
| 2 |
+ |
|
| 3 |
+The ./contrib/docker-device-tool contains a tool to manipulate devicemapper thin-pool. |
|
| 4 |
+ |
|
| 5 |
+Compile |
|
| 6 |
+======== |
|
| 7 |
+ |
|
| 8 |
+ $ make shell |
|
| 9 |
+ ## inside build container |
|
| 10 |
+ $ go build contrib/docker-device-tool/device_tool.go |
|
| 11 |
+ |
|
| 12 |
+ # if devicemapper version is old and compliation fails, compile with `libdm_no_deferred_remove` tag |
|
| 13 |
+ $ go build -tags libdm_no_deferred_remove contrib/docker-device-tool/device_tool.go |
| ... | ... |
@@ -397,6 +397,172 @@ there are two key directories. The `/var/lib/docker/devicemapper/mnt` directory |
| 397 | 397 |
image layer and container snapshot. The files contain metadata about each |
| 398 | 398 |
snapshot in JSON format. |
| 399 | 399 |
|
| 400 |
+## Increase capacity on a running device |
|
| 401 |
+ |
|
| 402 |
+You can increase the capacity of the pool on a running thin-pool device. This is |
|
| 403 |
+useful if the data's logical volume is full and the volume group is at full |
|
| 404 |
+capacity. |
|
| 405 |
+ |
|
| 406 |
+### For a loop-lvm configuration |
|
| 407 |
+ |
|
| 408 |
+In this scenario, the thin pool is configured to use `loop-lvm` mode. To show the specifics of the existing configuration use `docker info`: |
|
| 409 |
+ |
|
| 410 |
+```bash |
|
| 411 |
+$ sudo docker info |
|
| 412 |
+Containers: 0 |
|
| 413 |
+ Running: 0 |
|
| 414 |
+ Paused: 0 |
|
| 415 |
+ Stopped: 0 |
|
| 416 |
+Images: 2 |
|
| 417 |
+Server Version: 1.11.0-rc2 |
|
| 418 |
+Storage Driver: devicemapper |
|
| 419 |
+ Pool Name: docker-8:1-123141-pool |
|
| 420 |
+ Pool Blocksize: 65.54 kB |
|
| 421 |
+ Base Device Size: 10.74 GB |
|
| 422 |
+ Backing Filesystem: ext4 |
|
| 423 |
+ Data file: /dev/loop0 |
|
| 424 |
+ Metadata file: /dev/loop1 |
|
| 425 |
+ Data Space Used: 1.202 GB |
|
| 426 |
+ Data Space Total: 107.4 GB |
|
| 427 |
+ Data Space Available: 4.506 GB |
|
| 428 |
+ Metadata Space Used: 1.729 MB |
|
| 429 |
+ Metadata Space Total: 2.147 GB |
|
| 430 |
+ Metadata Space Available: 2.146 GB |
|
| 431 |
+ Udev Sync Supported: true |
|
| 432 |
+ Deferred Removal Enabled: false |
|
| 433 |
+ Deferred Deletion Enabled: false |
|
| 434 |
+ Deferred Deleted Device Count: 0 |
|
| 435 |
+ Data loop file: /var/lib/docker/devicemapper/devicemapper/data |
|
| 436 |
+ WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning. |
|
| 437 |
+ Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata |
|
| 438 |
+ Library Version: 1.02.90 (2014-09-01) |
|
| 439 |
+Logging Driver: json-file |
|
| 440 |
+[...] |
|
| 441 |
+``` |
|
| 442 |
+ |
|
| 443 |
+The `Data Space` values show that the pool is 100GiB total. This example extends the pool to 200GiB. |
|
| 444 |
+ |
|
| 445 |
+1. List the sizes of the devices. |
|
| 446 |
+ |
|
| 447 |
+ ```bash |
|
| 448 |
+ $ sudo ls -lh /var/lib/docker/devicemapper/devicemapper/ |
|
| 449 |
+ total 1.2G |
|
| 450 |
+ -rw------- 1 root root 100G Apr 14 08:47 data |
|
| 451 |
+ -rw------- 1 root root 2.0G Apr 19 13:27 metadata |
|
| 452 |
+ ``` |
|
| 453 |
+ |
|
| 454 |
+2. Truncate `data` file to 200GiB. |
|
| 455 |
+ |
|
| 456 |
+ ```bash |
|
| 457 |
+ $ sudo truncate -s 214748364800 /var/lib/docker/devicemapper/devicemapper/data |
|
| 458 |
+ ``` |
|
| 459 |
+ |
|
| 460 |
+3. Verify the file size changed. |
|
| 461 |
+ |
|
| 462 |
+ ```bash |
|
| 463 |
+ $ sudo ls -lh /var/lib/docker/devicemapper/devicemapper/ |
|
| 464 |
+ total 1.2G |
|
| 465 |
+ -rw------- 1 root root 200G Apr 14 08:47 data |
|
| 466 |
+ -rw------- 1 root root 2.0G Apr 19 13:27 metadata |
|
| 467 |
+ ``` |
|
| 468 |
+ |
|
| 469 |
+4. Reload data loop device |
|
| 470 |
+ |
|
| 471 |
+ ```bash |
|
| 472 |
+ $ sudo blockdev --getsize64 /dev/loop0 |
|
| 473 |
+ 107374182400 |
|
| 474 |
+ $ sudo losetup -c /dev/loop0 |
|
| 475 |
+ $ sudo blockdev --getsize64 /dev/loop0 |
|
| 476 |
+ 214748364800 |
|
| 477 |
+ ``` |
|
| 478 |
+ |
|
| 479 |
+5. Reload devicemapper thin pool. |
|
| 480 |
+ |
|
| 481 |
+ a. Get the pool name first. |
|
| 482 |
+ |
|
| 483 |
+ $ sudo dmsetup status | grep pool |
|
| 484 |
+ docker-8:1-123141-pool: 0 209715200 thin-pool 91 422/524288 18338/1638400 - rw discard_passdown queue_if_no_space - |
|
| 485 |
+ |
|
| 486 |
+ The name is the string before the colon. |
|
| 487 |
+ |
|
| 488 |
+ b. Dump the device mapper table first. |
|
| 489 |
+ |
|
| 490 |
+ $ sudo dmsetup table docker-8:1-123141-pool |
|
| 491 |
+ 0 209715200 thin-pool 7:1 7:0 128 32768 1 skip_block_zeroing |
|
| 492 |
+ |
|
| 493 |
+ c. Calculate the real total sectors of the thin pool now. |
|
| 494 |
+ |
|
| 495 |
+ Change the second number of the table info (i.e. the number of sectors) to reflect the new number of 512 byte sectors in the disk. For example, as the new loop size is 200GiB, change the second number to 419430400. |
|
| 496 |
+ |
|
| 497 |
+ d. Reload the thin pool with the new sector number |
|
| 498 |
+ |
|
| 499 |
+ $ sudo dmsetup suspend docker-8:1-123141-pool && sudo dmsetup reload docker-8:1-123141-pool --table '0 419430400 thin-pool 7:1 7:0 128 32768 1 skip_block_zeroing' && sudo dmsetup resume docker-8:1-123141-pool |
|
| 500 |
+ |
|
| 501 |
+#### The device_tool |
|
| 502 |
+ |
|
| 503 |
+The Docker's projects `contrib` directory contains not part of the core |
|
| 504 |
+distribution. These tools that are often useful but can also be out-of-date. <a |
|
| 505 |
+href="https://goo.gl/wNfDTi">In this directory, is the `device_tool.go`</a> |
|
| 506 |
+which you can also resize the loop-lvm thin pool. |
|
| 507 |
+ |
|
| 508 |
+To use the tool, compile it first. Then, do the following to resize the pool: |
|
| 509 |
+ |
|
| 510 |
+```bash |
|
| 511 |
+$ ./device_tool resize 200GB |
|
| 512 |
+``` |
|
| 513 |
+ |
|
| 514 |
+### For a direct-lvm mode configuration |
|
| 515 |
+ |
|
| 516 |
+In this example, you extend the capacity of a running device that uses the |
|
| 517 |
+`direct-lvm` configuration. This example assumes you are using the `/dev/sdh1` |
|
| 518 |
+disk partition. |
|
| 519 |
+ |
|
| 520 |
+1. Extend the volume group (VG) `vg-docker`. |
|
| 521 |
+ |
|
| 522 |
+ ```bash |
|
| 523 |
+ $ sudo vgextend vg-docker /dev/sdh1 |
|
| 524 |
+ Volume group "vg-docker" successfully extended |
|
| 525 |
+ ``` |
|
| 526 |
+ |
|
| 527 |
+ Your volume group may use a different name. |
|
| 528 |
+ |
|
| 529 |
+2. Extend the `data` logical volume(LV) `vg-docker/data` |
|
| 530 |
+ |
|
| 531 |
+ ```bash |
|
| 532 |
+ $ sudo lvextend -l+100%FREE -n vg-docker/data |
|
| 533 |
+ Extending logical volume data to 200 GiB |
|
| 534 |
+ Logical volume data successfully resized |
|
| 535 |
+ ``` |
|
| 536 |
+ |
|
| 537 |
+3. Reload devicemapper thin pool. |
|
| 538 |
+ |
|
| 539 |
+ a. Get the pool name. |
|
| 540 |
+ |
|
| 541 |
+ $ sudo dmsetup status | grep pool |
|
| 542 |
+ docker-253:17-1835016-pool: 0 96460800 thin-pool 51593 6270/1048576 701943/753600 - rw no_discard_passdown queue_if_no_space |
|
| 543 |
+ |
|
| 544 |
+ The name is the string before the colon. |
|
| 545 |
+ |
|
| 546 |
+ b. Dump the device mapper table. |
|
| 547 |
+ |
|
| 548 |
+ $ sudo dmsetup table docker-253:17-1835016-pool |
|
| 549 |
+ 0 96460800 thin-pool 252:0 252:1 128 32768 1 skip_block_zeroing |
|
| 550 |
+ |
|
| 551 |
+ c. Calculate the real total sectors of the thin pool now. we can use `blockdev` to get the real size of data lv. |
|
| 552 |
+ |
|
| 553 |
+ Change the second number of the table info (i.e. the number of sectors) to |
|
| 554 |
+ reflect the new number of 512 byte sectors in the disk. For example, as the |
|
| 555 |
+ new data `lv` size is `264132100096` bytes, change the second number to |
|
| 556 |
+ `515883008`. |
|
| 557 |
+ |
|
| 558 |
+ $ sudo blockdev --getsize64 /dev/vg-docker/data |
|
| 559 |
+ 264132100096 |
|
| 560 |
+ |
|
| 561 |
+ d. Then reload the thin pool with the new sector number. |
|
| 562 |
+ |
|
| 563 |
+ $ sudo dmsetup suspend docker-253:17-1835016-pool && sudo dmsetup reload docker-253:17-1835016-pool --table '0 515883008 thin-pool 252:0 252:1 128 32768 1 skip_block_zeroing' && sudo dmsetup resume docker-253:17-1835016-pool |
|
| 564 |
+ |
|
| 565 |
+ |
|
| 400 | 566 |
## Device Mapper and Docker performance |
| 401 | 567 |
|
| 402 | 568 |
It is important to understand the impact that allocate-on-demand and |