README
¶
journal 
Package journal is intended to be a very simple, but somewhat flexible logging
facility for golang services.
Main attributes:
- Selection of relevant columns for local logs
- Built-in or custom error messages
- Output to file and optionally stdout
- File rotation (daily, weekly, monthly, annually)
- Old log compression
- Remote logging/mirroring via grpc
- Tab-delimited or JSON-formatted output (output to stdout is always tab-delimited)
Logging locally
import log "github.com/vaitekunas/log"
logger := log.New(&log.Config{
})
notify := logger.Logfunc("Caller 1")
notify(0, "This is a simple message")
if err := notify(1, "Something went wrong: %s","Some general error"); err != nil {
notify(999, "Yup, things really went south: %s (%s)", "Should exit the program now","Nothing to be done here")
}
go func(){
notify := logger.Logfunc("Coroutine 1")
notify(0, "Starting coroutine")
notify(3, "")
}()
<- time.After(500*time.Millisecond)
notify(0, "Exiting")
Starting a remote logging facility and logging remotely
Mirroring local logs remotely is a nice way of aggregating information about a set of your services/instances in one place.
logserver start-service \
--host=127.0.0.1 --port=37746 \
--folder=$HOME/logs/ --file=myservice --stdout=true \
--rotation=daily --compress=true \
--cert=$HOME/logger/cert.pem --key=$HOME/logger/key.pem
--tokens=$HOME/logger/tokens.db
Authentication token management
In order to accept incoming connections we need to create authentication tokens for each service/instance of the service we wish to allow to connect to this server:
logserver add-token myservice myinstance
This creates a random 64-character token that the myinstance instance of
service myservice can use to connect to this server. We can also retrieve the
token for the instance:
logserver show-token myservice myinstance
or all instances of a service:
logserver show-tokens myservice
tokens can also be revoked:
logserver revoke-token myservice myinstance
We can also revoke all tokens and close all incoming connections for a given service:
logserver revoke-tokens myservice
revoking the token(s) closes any associated sessions and denies future connections.
Remote logging
import log "github.com/vaitekunas/log"
logger := log.New(&log.Config{
})
notify := logger.Logfunc("Caller 1")
// Connect to the log server
host := "127.0.0.1"
port := 37746
token := "8018A51319ADA07AF801595E18EABE438FF5C330241661B93C9C32ED36D60C00"
if err:= logger.Connect(host, port, instance, token, 5*time.Second); err != nil {
notify(1, "Could not connect to log server: %s", err.Error())
}else{
notify(0, "Connection to logserver (%s:%d) established.", host, port)
}
Local output:
DATA
Remote output:
DATA
The logging facility server will write all the incoming messages to the same file (and optionally stdout). In the logfile all the columns are present, even though incoming messages might have selected differing columns.
Remote logging is done asynchronously. The request will timeout according to the defined timeout duration.
Build
log imports the logrpc subpackage, which is generated from a protobuf definition,
so that in order to build log you first need to generate the golang stubs for it:
protoc --go_out=plugins=grpc:. protobuf/log.proto
TODO
- Implement
journal.ConnectToKafka
Documentation
¶
Overview ¶
Package journal implements a very simple logging facility with only one shared logging method, namely "Log".
Index ¶
- Constants
- func ConnectToJournald(host string, port int, service, instance, token string, timeout time.Duration) (io.WriteCloser, error)
- func ConnectToKafka(host string, port int, topic string) (io.WriteCloser, error)
- type Code
- type Config
- type Logger
- func (l *Logger) AddDestination(name string, writer io.Writer) error
- func (l *Logger) ListDestinations() []string
- func (l *Logger) Log(caller string, code int, msg string, format ...interface{}) error
- func (l *Logger) LogFields(caller string, code int, msg map[string]interface{}) error
- func (l *Logger) NewCaller(caller string) func(int, string, ...interface{}) error
- func (l *Logger) NewCallerCode(caller string, code int) func(string, ...interface{}) error
- func (l *Logger) Quit()
- func (l *Logger) RawEntry(entry map[int64]string) error
- func (l *Logger) RemoveDestination(name string) error
- func (l *Logger) UseCustomCodes(codes map[int]Code)
Examples ¶
Constants ¶
const ( ROT_NONE = 0 ROT_DAILY = 1 ROT_WEEKLY = 2 ROT_MONTHLY = 3 ROT_ANNUALLY = 4 )
File rotation frequency
const ( OUT_FILE = 0 OUT_STDOUT = 1 OUT_FILE_AND_STDOUT = 2 )
Output selection
const ( COL_DATE_YYMMDD = 0 COL_DATE_YYMMDD_HHMMSS = 1 COL_DATE_YYMMDD_HHMMSS_NANO = 2 COL_TIMESTAMP = 3 COL_SERVICE = 4 COL_INSTANCE = 5 COL_CALLER = 6 COL_MSG_TYPE_SHORT = 7 COL_MSG_TYPE_INT = 8 COL_MSG_TYPE_STR = 9 COL_MSG = 10 COL_FILE = 11 COL_LINE = 12 )
Log columns
Variables ¶
This section is empty.
Functions ¶
func ConnectToJournald ¶
func ConnectToJournald(host string, port int, service, instance, token string, timeout time.Duration) (io.WriteCloser, error)
ConnectToJournald connects to a log server backend
func ConnectToKafka ¶
ConnectToKafka connects to a kafka backend as a producer
Types ¶
type Code ¶
Code contains a single message type with an indicator of whether this message should be treated as an error.
type Config ¶
type Config struct {
Service string // Service name
Instance string // Instance name
Folder string // Folder to store logfiles (can be empty if logging to stdout only)
Filename string // Filename of the logfiles (without date suffix and file extension. Can be empty if logging to stdout only)
Rotation int // Logfile rotation frequency
Out int // Logger output type
Headers bool // Should the logfile contain column headers?
JSON bool // Should each entry be written as a JSON-formatted string?
Compress bool // Should old logfiles be compressed?
Columns []int64 // List of relevant columns (can be empty if default columns should be used)
}
Config contains all the necessary settings to create a new local logging facility
type Logger ¶
type Logger struct {
// contains filtered or unexported fields
}
Logger is the main loggger struct
func (*Logger) AddDestination ¶
AddDestination adds a (remote) destination to send logs to
func (*Logger) ListDestinations ¶
ListDestinations lists all (remote) destinations
func (*Logger) Log ¶
Log logs a simple message and returns nil or error, depending on the code
Example ¶
Simple example of local logging
package main
import (
"fmt"
"io/ioutil"
"os"
"testing"
)
// Test/example setup
func setup(t *testing.T) (tempdir string, teardown func()) {
dir, err := ioutil.TempDir(os.Getenv("HOME"), "example")
if err != nil {
t.Errorf("Could not create tempdir: %s", err.Error())
}
return dir, func() {
os.RemoveAll(dir)
}
}
// Simple example of local logging
func main() {
// Create temporary folder and teardown function
tempdir, teardown := setup(&testing.T{})
defer teardown()
// Instantiate logger
logger, err := New(&Config{
Service: "MyService",
Instance: "MyInstance",
Folder: tempdir,
Filename: "myservice",
Rotation: ROT_DAILY,
Out: OUT_FILE_AND_STDOUT,
Headers: true,
JSON: false,
Compress: true,
Columns: []int64{}, // Use default columns
})
if err != nil {
fmt.Printf("Could not start logger: %s", err.Error())
os.Exit(1)
}
// Log messages
notify := logger.NewCaller("Example 1")
notify(0, "Hello, World!")
}
func (*Logger) LogFields ¶
LogFields encodes the message (not the whole log) in JSON and writes to log
func (*Logger) NewCallerCode ¶
NewCallerCode is a wrapper for the Logger.fullog function
func (*Logger) RawEntry ¶
RawEntry writes a raw log entry (map of strings) into the ledger. The raw entry must contain columns COL_DATE_YYMMDD_HHMMSS_NANO to COL_LINE
func (*Logger) RemoveDestination ¶
RemoveDestination removes a (remote) destination to send logs to
func (*Logger) UseCustomCodes ¶
UseCustomCodes Replaces loggers default message codes with custom ones
Source Files
Directories