Fluentd v1.0

  • Logging is a critical component on production environments that allow us to perform monitoring and data analysis. While applications runs at scale, the Logging layer also needs to scale and year over year we see new challenges that needs to be solved from different angles such as parsing, performance, log enrichment (metadata), filtering and so on.

    Fluentd was born to solve Logging problems as a whole, not only for standalone applications but also for distributed architectures where each running application and system have their own way to solve logging, integration between all components and the ability to move data from one place to another in a secure and reliable way was a requirement from day one… and it continue to be as of today. That’s why it has been adopted by thousands of companies and thanks to it growing community around the world demonstrate that Fluentd became the Industry Standard for Logging.

    Fast iterations and solving the problem better than yesterday is our mantra, that’s why the global team of maintainers is proud to announce that Fluentd v1.0 has been released!, this is a big milestone for everyone around the Fluentd Ecosystem which means much more that a number, it’s a maturity resulting of years of work in the community where feedback, ideas and general testing have been the roots for it growth. So thank all you who have been involved on this process!

    What’s Hot in Fluentd v1.0 ?

    Fluentd v1.0 built on top of v0.14-stable, some of the biggest changes on this new series are:

    • Core:
      • Multiprocess Workers: take advantage of SMP systems
      • Sub-second time resolution: all log records now have a granular time resolution
    • Portability: Windows Support has finally arrived to Fluentd.
    • New plugins API: Our biggest contributions to Fluentd ecosystem is through plugins, with more than 700 plugins available made by the community we have focused into improve the developer experience.
    • Data Management: new internal buffers can optionally enable compressions to save disk space.
    • Networking:
      • New Fluentd Forward Protocol v1: includes authentication using shared keys and authorizations through username/password.
      • Native Transport-Layer-Security (TLS) support

    These changes are not the only ones, there many improvements around performance, portability and flexibility for data management.

    Monitoring and Data Streaming

    Fluentd is more than a project, it’s a full ecosystem and integration with third party components is fundamental, that’s why as part of our Fluentd v1.0 release we are proud to announce also the continuous investment in integration in Prometheus (monitoring) and Apache Kafka (data streaming) within many others.

    Prometheus makes easier monitoring and having Fluentd provide native support for it has been in high demand in the last time and we are happy to announce that fluent-plugin-prometheus is now officially part of Fluentd Ecosystem hosted on CNCF Fluent organization on Github.

    Data streaming is such important as monitoring, and companies around are looking for ways to integrate more components where Logging can be a critical compoment of the data pipeline. Fluentd is getting better and better at it: Fluentd and Apache Kafka can talk each other smoothly and securely.

    What’s Next

    Next year we will continue working towards performance improvements and connectors that makes easier to hook Fluentd on any place. As well from an ecosystem perspective, Fluent Bit, our lightweight log processor will keep growing in terms of capabilities for cloud native environments such as like load balancing, persistent queues and monitoring within others.

    2018 will be an exciting journey, don’t hesitate to be part of it!

    Connect at CloudNativeCon + KubeCon

    If you are attending KubeCon + CloudNativeCon North America this week, please join us December 6th for a Fluentd Salon, our community will be presenting the following lightning talks:

    • “Moving Data at Scale” by Sadayuki Furuhashi (Fluentd Creator)
    • “Fluentd and Kafka” by Masahiro Nakagawa (Fluentd Maintainer)
    • “Fluentd and Prometheus” by Yuki Ito (Fluentd Maintainer)
    • “Fluent Bit” by Eduardo Silva (Fluent Bit Maintainer)
    • “Plugging into fluent-bit: how to use plugin templates to customize fluent-bit to serve your needs” by Yeni Capote, Samsung SDS

    we will also discuss Fluentd v1.0, roadmap and tools around the Fluent ecosystem plus an open space for discussions. Additionally, do not miss the additional sessions on Fluentd:

    Closing Thoughts

    Reaching v1.0 for us it’s a new begginning, we will continue working together with our end-users, community and companies around to make Fluentd better for 2018. If you are around CloudNativeCon+KubeCon don’t hesitate to contact us in our sessions or during the break ours, come and join us!, you can also help to solve the problem better than yesterday !

    Let’s keep improving the Logging Standard…

    The post Fluentd v1.0 appeared first on The Linux Foundation.


Tmux Commands

screen and tmux

A comparison of the features (or more-so just a table of notes for accessing some of those features) for GNU screen and BSD-licensed tmux.

The formatting here is simple enough to understand (I would hope). ^ means ctrl+, so ^x is ctrl+x. M- means meta (generally left-alt or escape)+, so M-x is left-alt+x

It should be noted that this is no where near a full feature-set of either group. This - being a cheat-sheet - is just to point out the most very basic features to get you on the road.

Trust the developers and manpage writers more than me. This document is originally from 2009 when tmux was still new - since then both of these programs have had many updates and features added (not all of which have been dutifully noted here).

Action tmux screen
start a new session tmux OR
tmux new OR
tmux new-session
re-attach a detached session tmux attach OR
tmux attach-session
re-attach an attached session (detaching it from elsewhere) tmux attach -d OR
tmux attach-session -d
screen -dr
re-attach an attached session (keeping it attached elsewhere) tmux attach OR
tmux attach-session
screen -x
detach from currently attached session ^b d OR
^b :detach
^a ^d OR
^a :detach
rename-window to newname ^b , <newname> OR
^b :rename-window <newn>
^a A <newname>
list windows ^b w ^a w
list windows in chooseable menu ^a "
go to window # ^b # ^a #
go to last-active window ^b l ^a ^a
go to next window ^b n ^a n
go to previous window ^b p ^a p
see keybindings ^b ? ^a ?
list sessions ^b s OR
tmux ls OR
tmux list-sessions
screen -ls
toggle visual bell ^a ^g
create another window ^b c ^a c
exit current shell/window ^d ^d
split window/pane horizontally ^b " ^a S
split window/pane vertically ^b % ^a |
switch to other pane ^b o ^a <tab>
kill the current pane ^b x OR (logout/^D)
collapse the current pane/split (but leave processes running) ^a X
cycle location of panes ^b ^o
swap current pane with previous ^b {
swap current pane with next ^b }
show time ^b t
show numeric values of panes ^b q
toggle zoom-state of current pane (maximize/return current pane) ^b z
break the current pane out of its window (to form new window) ^b !
re-arrange current panels within same window (different layouts) ^b [space]
Kill the current window (and all panes within) ^b killw [target-window]
  • Use the same script for updating/ upgrading

    Make sure to change the versions to the latest releases:

    #!/bin/bash set -e bpcver=4.2.1 bpcxsver=0.57 rsyncbpcver=

    Scroll through the script, know what you are doing.

    Uncomment the upgrade section(s) and comment out the install section(s)

    read more
  • Again running smartctl after all is said and done:

    smartctl --all /dev/sda

    ddrescue-smartctl-after-rescue.png ddrescue-smartctl-2.png

    Yet an old drive in itself, I run the wheels off of them, and monitor regularly as anyone should.

    read more