# Photon OS Logs On Photon OS, all the system logs except the installation logs and the cloud-init logs are written into the systemd journal. The `journalctl` command queries the contents of the systemd journal. The installation log files and the cloud-init log files reside in `/var/log`. If Photon OS is running on a virtual machine in a VMware hypervisor, the log file for the VMware tools, `vmware-vmsvc.log`, also resides in `/var/log`. ##Journalctl Journalctl is a utility to query and display logs from journald and systemd’s logging service. Since journald stores log data in a binary format instead of a plain text format, `journalctl` is the standard way of reading log messages processed by journald. Journald is a service provided by systemd. To see the staus of the daemon, run the following commands: ``` # systemctl status systemd-journald ● systemd-journald.service - Journal Service Loaded: loaded (/lib/systemd/system/systemd-journald.service; static; vendor preset: enabled) Active: active (running) since Tue 2020-04-07 14:33:41 CST; 2 days ago Docs: man:systemd-journald.service(8) man:journald.conf(5) Main PID: 943 (systemd-journal) Status: "Processing requests..." Tasks: 1 (limit: 4915) Memory: 18.0M CGroup: /system.slice/systemd-journald.service └─943 /lib/systemd/systemd-journald Apr 07 14:33:41 photon-4a0e7f2307d4 systemd-journald[943]: Journal started Apr 07 14:33:41 photon-4a0e7f2307d4 systemd-journald[943]: Runtime journal (/run/log/journal/b8cebc61a6cb446a968ee1d4c5bbbbd5) is 8.0M, max 1.5G, 1.5G free. Apr 07 14:33:41 photon-4a0e7f2307d4 systemd-journald[943]: Time spent on flushing to /var is 88.263ms for 1455 entries. Apr 07 14:33:41 photon-4a0e7f2307d4 systemd-journald[943]: System journal (/var/log/journal/b8cebc61a6cb446a968ee1d4c5bbbbd5) is 40.0M, max 4.0G, 3.9G free. root@photon-4a0e7f2307d4 [ ~ ]# ``` The following command are related to `journalctl`: - `journalctl` : This command displays all the logs after the system has booted up. `journalctl` splits the results into pages, similar to the `less` command in Linux. You can navigate using the arrow keys, the Page Up, Page Down keys or the Space bar. To quit navigation, press the **q** key. - `journalctl -b` : This command displays the logs for the current boot. The following commands pull logs based on a time range: - `journalctl --since "1 hour ago"` : This command displays the journal logs from the past 1 hour. - `journalctl --since "2 days ago"` : This command displays the logs generated in the past 2 days. - `journalctl --since "2020-03-25 00:00:00" --until "2020-04-09 00:00:00"` : This command displays the logs generated between the mentioned time frame. To traverse for logs in the reverse order, run the following command: - `journalctl -r` : This command displays the logs in reverse order. **Note**: If you add `-r` at the end of a command, the logs are displayed in the reverse order. For example: `journalctl -u unit.service -r` To pull logs related to a particular daemon, run the following command: - `journalctl -u unit.service` : This command displays logs for a specific service. mention the name of the service instead of `unit`. This command helps when a service is not behaving properly or when there are crash/core dumps. To see Journal logs by their priority, run the following command: - `journalctl -p "emerg".."crit` : This command displays logs **emerg** to **critical**. For example: core dumps. Journalctl can print log messages to the console as they are added, like the Linux `tail` command. Add the `-f` switch to follow a specific service or daemon. ``` journalctl -u unit.service -f ``` To list the boots of the system, run the following command: ``` journalctl --list-boots ``` You can maintain the journalctl logs manually, by running the following `vacuum` commands: - `journalctl --vacuum-time=2d` : This command retains the logs from the last 2 days. - `journalctl --vacuum-size=500M` : This command helps retain logs with a maximum size of 500 MB. You can configure Journald using the conf file located at **/etc/systemd/journald.conf**. Run the following command to configure the file: ``` # cat /etc/systemd/journald.conf # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. # # Entries in this file show the compile time defaults. # You can change settings by editing this file. # Defaults can be restored by simply deleting this file. # # See journald.conf(5) for details. [Journal] #Storage=auto #Compress=yes #Seal=yes #SplitMode=uid #SyncIntervalSec=5m #RateLimitIntervalSec=30s #RateLimitBurst=10000 #SystemMaxUse= #SystemKeepFree= #SystemMaxFileSize= #SystemMaxFiles=100 #RuntimeMaxUse= #RuntimeKeepFree= #RuntimeMaxFileSize= #RuntimeMaxFiles=100 #MaxRetentionSec= #MaxFileSec=1month #ForwardToSyslog=no #ForwardToKMsg=no #ForwardToConsole=no #ForwardToWall=yes #TTYPath=/dev/console #MaxLevelStore=debug #MaxLevelSyslog=debug #MaxLevelKMsg=notice #MaxLevelConsole=info #MaxLevelWall=emerg #LineMax=48K root@photon-4a0e7f2307d4 [ ~ ]# ``` By default `rotate` is deactivate in Photon. Once the changes are made to the conf file, for the changes to take effect you must restart the systemd-journald by running the `systemctl restart systemd-journald` command. ##Cloud-init Logs Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialisation. If there are with the Cloud-init behaviour, we can debug them by looking at the logs. Run the following command to look at Cloud-init logs: ``` journalctl -u cloud-init ``` For better understanding/debugging, You can also look at logs from the following locations: - **/var/log/cloud-init.log** : This log contains information from each stage of Cloud-init. - **/var/log/cloud-init-output.log** : This log contains errors, warnings, etc.. ##Syslog Syslog is the general standard for logging system and program messages in the Linux environment. Photon provides the following two packages to support syslog: - **syslog-ng** : syslog-ng is syslog with some advanced next gen features. It supports TLS encryption, TCP for transport with other existing features. Configurations can be added to the **/etc/syslog-ng/syslog-ng.conf** file. - **rsyslog** : The official RSYSLOG website defines the utility as "the rocket-fast system for log processing". rsyslog supports some advanced features like relp, imfile, omfile, gnutls protocols. Configurations can be added to the **/etc/rsyslog.conf** file. You can configure the required TLS certificates by editing the conf file. ##Logs for RPMS on Photon Logs for a particular RPM can be checked in the following ways: - If the RPM provides a daemon, we can see the status of daemon by running `systemctl` command and check logs using `journactl -u <service name>` command. - For additional logs, check if a conf file is provided by the RPM by running the `rpm -ql <rpm name> | grep conf` command and find the file path of the log file. You can also check the **/var/log** folder.