GitList

Browse code

Fix worlddump log collection

All credit for figuring this out goes to frickler (and that was the hard
bit so thank you!). The worlddump files were not being collected because
they weren't in our log collection list. Add worlddump to this list
so that we collect these files.

One thing that makes this slightly complicated is the worlddump files
are named with a timestamp and we can't have globs in our collection
list. To address this we create a copy of the file with a -latest.txt
suffix. This gives us a deterministic file name for log collection
without using globs.

Note we do not use a symlink here because some jobs gzip their log files
(breaking symlinks) and others do not. This makes it painful to always
have a valid link. Not having a valid link can break log collection.

Hardlinks may be another option but simply making a copy is easier to
manage as you don't have to worry about links preexisting and the
dumpfiles are not that large.

Change-Id: I96ae5f5290546ad25ca434c1106c01354d2d053c

Jens Harbott authored on 2019/09/05 17:51:33
Showing 2 changed files

.zuul.yaml

History View file @ ce396d3

@@ -233,6 +233,7 @@
                              '{{ devstack_log_dir }}/devstacklog.txt': logs
                              '{{ devstack_log_dir }}/devstacklog.txt.summary': logs
                              '{{ devstack_log_dir }}/tcpdump.pcap': logs
                     +        '{{ devstack_log_dir }}/worlddump-latest.txt': logs
                              '{{ devstack_full_log}}': logs
                              '{{ stage_dir }}/verify_tempest_conf.log': logs
                              '{{ stage_dir }}/apache': logs

tools/worlddump.py

History View file @ ce396d3

@@ -25,6 +25,7 @@ from distutils import spawn
                      import fnmatch
                      import os
                      import os.path
                     +import shutil
                      import subprocess
                      import sys
@@ -248,6 +249,14 @@ def main():
                              compute_consoles()
                              guru_meditation_reports()
                              var_core()
                     +    # Singular name for ease of log retrieval
                     +    copyname = os.path.join(opts.dir, 'worlddump')
                     +    if opts.name:
                     +        copyname += '-' + opts.name
                     +    copyname += '-latest.txt'
                     +    # We make a full copy to deal with jobs that may or may not
                     +    # gzip logs breaking symlinks.
                     +    shutil.copyfile(fname, copyname)
                      if __name__ == '__main__':