[ art / civ / cult / cyb / diy / drg / feels / layer / lit / λ / q / r / sci / sec / tech / w / zzz ] archive provided by lainchan.jp

lainchan archive - /λ/ - 21890



File: 1486406277909.png (21.56 KB, 300x291, pretty_output.gif)

No.21890

Are there any guides or recommended readings that break down standards or outline guidelines for creating meaningful output from programs?

Striking a balance between output that can easily digested by other programs in a workflow ( piping output to another program for processing ) but keeping the output readable and meaningful to a human is a constant challenge.

There seems to be a lack of documentation on methods or ideas that pertain to finding this balance. Avoiding log files to store human readable output is ideal since that becomes a headache in itself when it comes to long running daemons.

Anyone else deal with these problems or have good documents to read?

  No.21896

One way is detecting whether the output is being piped somewhere or printed to stdout and adjusting your output accordingly.
Another, simpler and more consistent, option would be to introduce command line switches that allow the user to chose between easily parsable and pleasantly readable formats.

  No.21897

>Are there any guides or recommended readings that break down standards or outline guidelines for creating meaningful output from programs?
There's plenty to choose from, which is why most sytems have little cohesion.

>Striking a balance between output that can easily digested by other programs in a workflow ( piping output to another program for processing ) but keeping the output readable and meaningful to a human is a constant challenge.

The obvious solution is to have two representations. Consider how you would do this in a normal programming system. You would have an output humans are expected to use, but you'll be passing around objects or the relevant representations to other parts of the same program or even other programs.

>There seems to be a lack of documentation on methods or ideas that pertain to finding this balance. Avoiding log files to store human readable output is ideal since that becomes a headache in itself when it comes to long running daemons.

>Anyone else deal with these problems or have good documents to read?
Consider if it's actually useful to have programs that parse the textual output of other programs. Contrary to the thinking, this actually creates a strong dependence on output format. Try to write programs that pass machine representations around, rather than parse and serialize text constantly. If you must use text, use a recognized format rather than something special your program produces, such as S-expressions, JSON, et al.

  No.21899

The right way is to use standard streams. Standard output receives your program's actual output and nothing but the output. Despite it's name, standard error is a side channel meant to receive all information meant for the user.

So if you make a program that performs a computation in several steps and returns the answer as a number, the number goes straight to standard output while any and all diagnostic messages like "Doing something... done" go to standard error.

This is good since the user can choose to redirect them separately and to destinations of his choice. Since silence is golden, normally only the output is expected. Unless an error is the output, the standard error stream is better left alone unless the user switches on verbose mode.

In addition to the above, one may also determine whether stream is connected to a terminal. Interactive users may benefit from additional output. Be sure to allow the user to override your detection, though.

If your output is meant to be consumed by machines rather than humans, it pays to output data in a machine readable format since it will greatly facillitate integration between systems. The newline-delimited variant of JSON is a favorite of mine.

Even diagnostic messages may be encoded in JSON -- why not? It makes no sense to serialize structured data into a text message if it's going to be parsed back into a data structure again. However, it's difficult to pass around binary data structures without serializing them. JSON isn't perfect but it has saved me a lot of effort in writing parsers for log messages.

  No.21908

>>21890

You basically have 2 options, data is in a certain order or data is not in order, but you scan through it looking for keywords in a data structure of some type. Probably dictionary.

First one is obviously faster, Second one is more user friendly

>Avoiding log files to store human readable output is ideal since that becomes a headache in itself when it comes to long running daemons.


You can have a program which converts machine readable to human readable. Or even combine log entries, if the last 100 entries have been the same, combine them, save yourself 99 lines

  No.21943

>>21896
>>21897
>>21899
>>21908
(OP here)
Interesting responses. I've always hesitated to use stderr as a way to write output to the user while preserving stdout so data can be easily digested by other *nix tools, not sure why.

Good points about not trying to constantly process text if i can better represent the data as another data structure. I'll keep this in mind when i know my output is going to another one of my own programs instead of generic unix tools.

A coworker also recommended using [] and () marks and tabbing inside my human output to help delineate between errors [!] additional info [+] the next operation [.] etc. While wrapping useful information in ().

[!] Error: Failed to find cache at (/path/to/file)

Hopefully sticking this output to stderr in all my programs would allow me to add in quiet flags and log path flags, then if all my logs are the same format i can write a log parser.

Thanks for the suggestions everyone.