s3cmd.1
dd4fec78
 .TH s3cmd 1
 .SH NAME
01a80524
 s3cmd \- tool for managing Amazon S3 storage space and Amazon CloudFront content delivery network
dd4fec78
 .SH SYNOPSIS
 .B s3cmd
 [\fIOPTIONS\fR] \fICOMMAND\fR [\fIPARAMETERS\fR]
 .SH DESCRIPTION
 .PP
 .B s3cmd
 is a command line client for copying files to/from 
 Amazon S3 (Simple Storage Service) and performing other
 related tasks, for instance creating and removing buckets,
 listing objects, etc.
 .PP
 .B s3cmd
 can do several \fIactions\fR specified by the following \fIcommands\fR.
 .TP
 \fBmb\fR \fIs3://BUCKET\fR
 Make bucket
 .TP
 \fBrb\fR \fIs3://BUCKET\fR
 Remove bucket
 .TP
 \fBls\fR \fI[s3://BUCKET[/PREFIX]]\fR
 List objects or buckets
 .TP
 \fBla\fR
 List all object in all buckets
 .TP
 \fBput\fR \fIFILE [FILE...] s3://BUCKET[/PREFIX]\fR
7393bdba
 Put file into bucket (i.e. upload to S3)
dd4fec78
 .TP
 \fBget\fR \fIs3://BUCKET/OBJECT LOCAL_FILE\fR
7393bdba
 Get file from bucket (i.e. download from S3)
dd4fec78
 .TP
 \fBdel\fR \fIs3://BUCKET/OBJECT\fR
 Delete file from bucket
f298b348
 .TP
7393bdba
 \fBsync\fR \fILOCAL_DIR s3://BUCKET[/PREFIX]\fR
29893afc
 Backup a directory tree to S3
 .TP
 \fBsync\fR \fIs3://BUCKET[/PREFIX] LOCAL_DIR\fR
 Restore a tree from S3 to local directory
7393bdba
 .TP
01a80524
 \fBcp\fR \fIs3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]\fR, \fBmv\fR \fIs3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]\fR 
7c2e60f1
 Make a copy of a file (\fIcp\fR) or move a file (\fImv\fR). 
 Destination can be in the same bucket with a different name
7d0ac8ee
 or in another bucket with the same or different name.
7c2e60f1
 Adding \fI\-\-acl\-public\fR will make the destination object 
 publicly accessible (see below).
01a80524
 .TP
 \fBsetacl\fR \fIs3://BUCKET[/OBJECT]\fR
 Modify \fIAccess control list\fI for Bucket or Files. Use with 
 \fI\-\-acl\-public\fR or \fI\-\-acl\-private\fR
7d0ac8ee
 .TP 
7393bdba
 \fBinfo\fR \fIs3://BUCKET[/OBJECT]\fR
 Get various information about a Bucket or Object
 .TP
f298b348
 \fBdu\fR \fI[s3://BUCKET[/PREFIX]]\fR
bf42f420
 Disk usage \- amount of data stored in S3
dd4fec78
 
01a80524
 .PP
 Commands for CloudFront management
 .TP
 \fBcflist\fR
 List CloudFront distribution points
 .TP
 \fBcfinfo\fR [\fIcf://DIST_ID\fR]
 Display CloudFront distribution point parameters
 .TP
 \fBcfcreate\fR \fIs3://BUCKET\fR
 Create CloudFront distribution point
 .TP
 \fBcfdelete\fR \fIcf://DIST_ID\fR
 Delete CloudFront distribution point
 .TP
 \fBcfmodify\fR \fIcf://DIST_ID\fR
 Change CloudFront distribution point parameters
 
dd4fec78
 .SH OPTIONS
 .PP
 Some of the below specified options can have their default 
 values set in 
 .B s3cmd
 config file (by default $HOME/.s3cmd). As it's a simple text file 
 feel free to open it with your favorite text editor and do any
01a80524
 changes you like. 
dd4fec78
 .PP
01a80524
 \fIConfig file related options.\fR
dd4fec78
 .TP
bf42f420
 \fB\-\-configure\fR
dd4fec78
 Invoke interactive (re)configuration tool. Don't worry, you won't 
 lose your settings on subsequent runs.
 .TP
bf42f420
 \fB\-c\fR FILE, \fB\-\-config\fR=FILE
dd4fec78
 Config file name. Defaults to $HOME/.s3cfg
 .TP
bf42f420
 \fB\-\-dump\-config\fR
dd4fec78
 Dump current configuration after parsing config files
 and command line options and exit.
 .PP
01a80524
 \fIOptions specific for \fIfile transfer commands\fR (\fBsync\fR, \fBput\fR and \fBget\fR):
 .TP
 \fB\-n\fR, \fB\-\-dry\-run\fR
 Only show what should be uploaded or downloaded but don't actually do it. May still perform S3 requests to get bucket listings and other in
 formation though.
7393bdba
 .TP
 \fB\-\-delete\-removed\fR
29893afc
 Delete remote objects with no corresponding local file when \fIsync\fRing \fBto\fR S3 or delete local files with no corresponding object in S3 when \fIsync\fRing \fBfrom\fR S3.
7393bdba
 .TP
 \fB\-\-no\-delete\-removed\fR
01a80524
 Don't delete remote objects. Default for \fIsync\fR command.
7393bdba
 .TP
 \fB\-p\fR, \fB\-\-preserve\fR
01a80524
 Preserve filesystem attributes (mode, ownership, timestamps). Default for \fIsync\fR command.
7393bdba
 .TP
 \fB\-\-no\-preserve\fR
 Don't store filesystem attributes with uploaded files.
ee852c0a
 .TP
2d7d5543
 \fB\-\-exclude GLOB\fR
01a80524
 Exclude files matching GLOB (a.k.a. shell-style wildcard) from \fIsync\fI. See FILE TRANSFERS section and \fIhttp://s3tools.org/s3cmd-sync\fR for more information.
ee852c0a
 .TP
 \fB\-\-exclude\-from FILE\fR
2d7d5543
 Same as \-\-exclude but reads GLOBs from the given FILE instead of expecting them on the command line.
 .TP
 \fB\-\-rexclude REGEXP\fR
 Same as \-\-exclude but works with REGEXPs (Regular expressions).
 .TP
 \fB\-\-rexclude\-from FILE\fR
 Same as \-\-exclude\-from but works with REGEXPs.
ee852c0a
 .TP
01a80524
 \fB\-\-include=GLOB\fR, \fB\-\-include\-from=FILE\fR, \fB\-\-rinclude=REGEXP\fR, \fB\-\-rinclude\-from=FILE\fR
 Filenames and paths matching GLOB or REGEXP will be included even if previously excluded by one of \-\-(r)exclude(\-from) patterns
dd4fec78
 .TP
9197e62e
 \fB\-\-continue\fR
01a80524
 Continue getting a partially downloaded file (only for \fIget\fR command). This comes handy once download of a large file, say an ISO image, from a S3 bucket fails and a partially downloaded file is left on the disk. Unfortunately \fIput\fR command doesn't support restarting of failed upload due to Amazon S3 limitations.
dd4fec78
 .TP
01a80524
 \fB\-\-skip\-existing\fR
 Skip over files that exist at the destination (only for \fIget\fR and \fIsync\fR commands).
18c4a150
 .TP
bf42f420
 \fB\-m\fR MIME/TYPE, \fB\-\-mime\-type\fR=MIME/TYPE
 Default MIME\-type to be set for objects stored.
dd4fec78
 .TP
bf42f420
 \fB\-M\fR, \fB\-\-guess\-mime\-type\fR
 Guess MIME\(hytype of files by their extension. Falls
 back to default MIME\(hyType as specified by \fB\-\-mime\-type\fR
dd4fec78
 option
 .TP
01a80524
 \fB\-\-add\-header=NAME:VALUE\fR
 Add a given HTTP header to the upload request. Can be used multiple times with different header names. For instance set 'Expires' or 'Cache-Control' headers (or both) using this options if you like.
 .TP
 \fB\-P\fR, \fB\-\-acl\-public\fR
 Store objects with permissions allowing read for anyone. See \fIhttp://s3tools.org/s3cmd-public\fR for details and hints for storing publicly accessible files.
 .TP
 \fB\-\-acl\-private\fR
 Store objects with default ACL allowing access for you only.
 .TP
 \fB\-e\fR, \fB\-\-encrypt\fR
 Use GPG encryption to protect stored objects from unauthorized access. See \fIhttp://s3tools.org/s3cmd-public\fR for details about encryption.
 .TP
 \fB\-\-no\-encrypt\fR
 Don't encrypt files.
 .PP
 \fIOptions for CloudFront commands\fR:
 .PP
 See \fIhttp://s3tools.org/s3cmd-cloudfront\fR for more details.
 .TP
 \fB\-\-enable\fR
 Enable given CloudFront distribution (only for \fIcfmodify\fR command)
 .TP
 \fB\-\-disable\fR
 Enable given CloudFront distribution (only for \fIcfmodify\fR command)
 .TP
 \fB\-\-cf\-add\-cname=CNAME\fR
 Add given CNAME to a CloudFront distribution (only for \fIcfcreate\fR and \fIcfmodify\fR commands)
 .TP
 \fB\-\-cf\-remove\-cname=CNAME\fR
 Remove given CNAME from a CloudFront distribution (only for \fIcfmodify\fR command)
 .TP
 \fB\-\-cf\-comment=COMMENT\fR
 Set COMMENT for a given CloudFront distribution (only for \fIcfcreate\fR and \fIcfmodify\fR commands)
 .PP
 \fIOptions common for all commands\fR (where it makes sense indeed):
 .TP
 \fB\-r\fR, \fB\-\-recursive\fR
 Recursive upload, download or removal. When used with \fIdel\fR it can
 remove all the files in a bucket.
 .TP
 \fB\-f\fR, \fB\-\-force\fR
 Force overwrite and other dangerous operations. Can be used to remove 
 a non\-empty buckets with \fIs3cmd rb \-\-force s3://bkt\fR
 .TP
 \fB\-\-bucket\-location\fR=BUCKET_LOCATION
 Specify datacentre where to create the bucket. Possible values are \fIUS\fR (default) or \fIEU\fR.
 .TP
bf42f420
 \fB\-H\fR, \fB\-\-human\-readable\-sizes\fR
dd4fec78
 Print sizes in human readable form.
01a80524
 .TP
 \fB\-\-list\-md5\fR
 Include MD5 sums in bucket listings (only for \fIls\fR command).
dd4fec78
 .TP
25f6f8c9
 \fB\-\-progress\fR, \fB\-\-no\-progress\fR
 Display or don't display progress meter. When running on TTY (e.g. console or xterm) the default is to display progress meter. If not on TTY (e.g. output is redirected somewhere or running from cron) the default is to not display progress meter.
 .TP
01a80524
 \fB\-\-encoding=ENCODING\fR
 Override autodetected terminal and filesystem encoding (character set).
 .TP
bf42f420
 \fB\-v\fR, \fB\-\-verbose\fR
dd4fec78
 Enable verbose output.
 .TP
bf42f420
 \fB\-d\fR, \fB\-\-debug\fR
dd4fec78
 Enable debug output.
 .TP
bf42f420
 \fB\-h\fR, \fB\-\-help\fR
dd4fec78
 Show the help message and exit
 .TP
bf42f420
 \fB\-\-version\fR
dd4fec78
 Show
 .B s3cmd
 version and exit.
 
01a80524
 .SH FILE TRANSFERS
ee852c0a
 One of the most powerful commands of \fIs3cmd\fR is \fBs3cmd sync\fR used for 
01a80524
 synchronising complete directory trees to or from remote S3 storage. To some extent 
 \fBs3cmd put\fR and \fBs3cmd get\fR share a similar behaviour with \fBsync\fR.
ee852c0a
 .PP
 Basic usage common in backup scenarios is as simple as:
 .nf
01a80524
 	s3cmd sync /local/path/ s3://test-bucket/backup/
ee852c0a
 .fi
 .PP
 This command will find all files under /local/path directory and copy them 
 to corresponding paths under s3://test-bucket/backup on the remote side.
 For example:
 .nf
01a80524
 	/local/path/\fBfile1.ext\fR         \->  s3://bucket/backup/\fBfile1.ext\fR
 	/local/path/\fBdir123/file2.bin\fR  \->  s3://bucket/backup/\fBdir123/file2.bin\fR
ee852c0a
 .fi
01a80524
 .PP
 However if the local path doesn't end with a slash the last directory's name
 is used on the remote side as well. Compare these with the previous example:
 .nf
 	s3cmd sync /local/path s3://test-bucket/backup/
 .fi
 will sync:
 .nf
 	/local/\fBpath/file1.ext\fR         \->  s3://bucket/backup/\fBpath/file1.ext\fR
 	/local/\fBpath/dir123/file2.bin\fR  \->  s3://bucket/backup/\fBpath/dir123/file2.bin\fR
 .fi
 .PP
ee852c0a
 To retrieve the files back from S3 use inverted syntax:
 .nf
01a80524
 	s3cmd sync s3://test-bucket/backup/ /tmp/restore/
ee852c0a
 .fi
 that will download files:
 .nf
01a80524
 	s3://bucket/backup/\fBfile1.ext\fR         \->  /tmp/restore/\fBfile1.ext\fR       
 	s3://bucket/backup/\fBdir123/file2.bin\fR  \->  /tmp/restore/\fBdir123/file2.bin\fR
ee852c0a
 .fi
2d7d5543
 .PP
01a80524
 Without the trailing slash on source the behaviour is similar to 
 what has been demonstrated with upload:
 .nf
 	s3cmd sync s3://test-bucket/backup /tmp/restore/
 .fi
 will download the files as:
 .nf
 	s3://bucket/\fBbackup/file1.ext\fR         \->  /tmp/restore/\fBbackup/file1.ext\fR       
 	s3://bucket/\fBbackup/dir123/file2.bin\fR  \->  /tmp/restore/\fBbackup/dir123/file2.bin\fR
 .fi
 .PP
 All source file names, the bold ones above, are matched against \fBexclude\fR 
 rules and those that match are then re\-checked against \fBinclude\fR rules to see
 whether they should be excluded or kept in the source list.
 .PP
 For the purpose of \fB\-\-exclude\fR and \fB\-\-include\fR matching only the 
 bold file names above are used. For instance only \fBpath/file1.ext\fR is tested
 against the patterns, not \fI/local/\fBpath/file1.ext\fR
ee852c0a
 .PP
01a80524
 Both \fB\-\-exclude\fR and \fB\-\-include\fR work with shell-style wildcards (a.k.a. GLOB).
 For a greater flexibility s3cmd provides Regular-expression versions of the two exclude options 
 named \fB\-\-rexclude\fR and \fB\-\-rinclude\fR. 
 The options with ...\fB\-from\fR suffix (eg \-\-rinclude\-from) expect a filename as
 an argument. Each line of such a file is treated as one pattern.
ee852c0a
 .PP
01a80524
 There is only one set of patterns built from all \fB\-\-(r)exclude(\-from)\fR options
 and similarly for include variant. Any file excluded with eg \-\-exclude can 
 be put back with a pattern found in \-\-rinclude\-from list.
ee852c0a
 .PP
01a80524
 Run s3cmd with \fB\-\-dry\-run\fR to verify that your rules work as expected. 
 Use together with \fB\-\-debug\fR get detailed information
 about matching file names against exclude and include rules.
ee852c0a
 .PP
01a80524
 For example to exclude all files with ".jpg" extension except those beginning with a number use:
ee852c0a
 .PP
01a80524
 	\-\-exclude '*.jpg' \-\-rinclude '[0-9].*\.jpg'
 
 .SH SEE ALSO
 For the most up to date list of options run 
 .B s3cmd \-\-help
 .br
 For more info about usage, examples and other related info visit project homepage at
 .br
 .B http://s3tools.org
ee852c0a
 
dd4fec78
 .SH AUTHOR
 Written by Michal Ludvig <michal@logix.cz>
01a80524
 .SH CONTACT, SUPPORT
 Prefered way to get support is our mailing list:
 .I s3tools\-general@lists.sourceforge.net
dd4fec78
 .SH REPORTING BUGS
 Report bugs to 
01a80524
 .I s3tools\-bugs@lists.sourceforge.net
dd4fec78
 .SH COPYRIGHT
01a80524
 Copyright \(co 2007,2008,2009 Michal Ludvig <http://www.logix.cz/michal>
dd4fec78
 .br
 This is free software.  You may redistribute copies of it under the terms of
 the GNU General Public License version 2 <http://www.gnu.org/licenses/gpl.html>.
 There is NO WARRANTY, to the extent permitted by law.