Pulsar (Linux) (pulsar.py)

Watch files and translate the changes into salt events

depends:
  • pyinotify Python module >= 0.9.5
Caution:

Using generic mask options like open, access, ignored, and closed_nowrite with reactors can easily cause the reactor to loop on itself. To mitigate this behavior, consider setting the disable_during_state_run flag to True in the beacon configuration.

hubblestack.extmods.modules.pulsar.process(configfile='salt://hubblestack_pulsar/hubblestack_pulsar_config.yaml', verbose=False)[source]

Watch the configured files

Example pillar config

beacons:
  pulsar:
    paths:
      - /var/cache/salt/minion/files/base/hubblestack_pulsar/hubblestack_pulsar_config.yaml
    refresh_interval: 300
    verbose: False

Example yaml config on fileserver (targeted by pillar)

/path/to/file/or/dir:
  mask:
    - open
    - create
    - close_write
  recurse: True
  auto_add: True
  exclude:
    - /path/to/file/or/dir/exclude1
    - /path/to/file/or/dir/exclude2
    - /path/to/file/or/dir/regex[\d]*$:
        regex: True
return:
  splunk:
    batch: True
  slack:
    batch: False  # overrides the global setting
checksum: sha256
stats: True
batch: True
contents_size: 20480
checksum_size: 104857600

Note that if batch: True, the configured returner must support receiving a list of events, rather than single one-off events.

The mask list can contain the following events (the default mask is create, delete, and modify):

  • access - File accessed
  • attrib - File metadata changed
  • close_nowrite - Unwritable file closed
  • close_write - Writable file closed
  • create - File created in watched directory
  • delete - File deleted from watched directory
  • delete_self - Watched file or directory deleted
  • modify - File modified
  • moved_from - File moved out of watched directory
  • moved_to - File moved into watched directory
  • move_self - Watched file moved
  • open - File opened

The mask can also contain the following options:

  • dont_follow - Don’t dereference symbolic links
  • excl_unlink - Omit events for children after they have been unlinked
  • oneshot - Remove watch after one event
  • onlydir - Operate only if name is directory
recurse:
Recursively watch directories under the named directory
auto_add:
Python inotify option, meaning: automatically start watching new directories that are created in a watched directory
watch_new_files:
when a new file is created in a watched dir, add a watch on the file (implied by watch_files below)
watch_files:
add explicit watches on all files (except excluded) under the named directory
exclude:
Exclude directories or files from triggering events in the watched directory. Can use regex if regex is set to True
contents:
Retrieve the contents of changed files based on checksums (which must be enabled)

If pillar/grains/minion config key hubblestack:pulsar:maintenance is set to True, then changes will be discarded.

hubblestack.extmods.modules.pulsar.top(topfile='salt://hubblestack_pulsar/top.pulsar', verbose=False)[source]

Execute pulsar using a top.pulsar file to decide which configs to use for this host.

The topfile should be formatted like this:

pulsar:
  '<salt compound match identifying host(s)>':
    - list.of.paths
    - using.dots.as.directory.separators

Paths in the topfile should be relative to salt://hubblestack_pulsar, and the .yaml should not be included.