Platforms
SMACK
- Why is the SMACK stack all the rage lately? (November 20, 2015)
- Data processing platforms architectures with SMACK: Spark, Mesos, Akka, Cassandra and Kafka (16 SEPTEMBER 2015)
- Streaming Analytics with Spark, Kafka, Cassandra and Akka (Oct 31, 2015)
- KillrWeather
- a reference application (which we are constantly improving) showing how to easily leverage and integrate Apache Spark, Apache Cassandra, and Apache Kafka for fast, streaming computations in asynchronous Akka event-driven environments.
Vagrant
- https://www.vagrantup.com/
- Desc. : Create and configure lightweight, reproducible, and portable development environments.
ROS
- http://www.ros.org/
- Desc. : a flexible framework for writing robot software.
- License :
Frameworks
Networking
Vert.x
- http://vertx.io/
- Desc. : a tool-kit for building reactive applications on the JVM.
- License :
- Sources : https://github.com/eclipse/vert.x
- Readings
Testing
Selenium
- http://docs.seleniumhq.org/
- Desc. : automates browsers
- License : Apache License, Version 2.0
- Readings
- Official documentation : http://docs.seleniumhq.org/docs/
- Java API
XMLUnit
- http://xmlunit.sourceforge.net/
- Desc. : JUnit and NUnit testing for XML
Mobile
Sencha Touch
- http://www.sencha.com/products/touch/
- Desc. : a high-performance HTML5 mobile application framework
- License : GPLv3 or commercial
- Readings
Libraries
XML
EXSLT
- http://exslt.org/
- Desc. : a community initiative to provide extensions to XSLT.
- License : ?
Jing
- https://code.google.com/p/jing-trang/
- Desc. : an application for validating an XML document against a RELAX NG schema, in either the XML or the compact syntax.
- License : New BSD License
- Readings
WSDL viewer
- https://code.google.com/p/wsdl-viewer/
- Desc. : transforms the WSDL into HTML.
- License : New BSD License
Networking
gRPC
- https://grpc.io/
- Desc. : A high performance, open-source universal RPC framework
- License : Apache License 2.0
- Written in :
- Sources : https://github.com/grpc/
Protocol Buffers
- https://code.google.com/p/protobuf/
- Desc. : a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
- License : BSD License
misc
CKEditor
- http://ckeditor.com/
- Desc. : a text editor to be used inside web pages.
- License : GPL, LGPL, MPL
markItUp
- http://markitup.jaysalvat.com/
- Desc. : allows you to turn any textarea into a markup editor such as Html, Textile, Wiki Syntax, Markdown, BBcode or even your own Markup system.
- License : MIT/GPL
Scintilla
- http://www.scintilla.org/
- Desc. : A free source code editing component for Win32, GTK+, and OS X
- License : permits use in any free project or commercial product.
Tools
IDE
NetBeans
- https://netbeans.org/
- Desc. : lets you quickly and easily develop Java desktop, mobile, and web applications, while also providing great tools for PHP and C/C++ developers.
- License : dual license consisting of the CDDL v1.0 and the GPL v2
IntelliJ IDEA
- https://www.jetbrains.com/idea/
- Desc. : intelligent IDE for Java and other technologies with an exceptional out-of-the-box feature set
- License :
Plugins
Plugin | Site | Description | Remarks |
---|---|---|---|
BashSupport | https://www.plugin-dev.com/project/bashsupport/ | provides an almost complete development environment to work with Bash scripts in IntelliJ platform products |
Modeling
Sirius
- http://www.eclipse.org/sirius/
- Desc. : an Eclipse project which allows you to easily create your own graphical modeling workbench by leveraging the Eclipse Modeling technologies, including EMF and GMF.
- License
SQL Developer Data Modeler
- http://www.oracle.com/technetwork/developer-tools/datamodeler/overview/
- Desc. : create, browse and edit, logical, relational, physical, multi-dimensional, and data type models.
- License :
- Features supported or not supported
Feature | Supporting | Remarks |
---|---|---|
Defining AK(Alternative Key) | ||
Defining View | Yes | |
Defining FK from AK | ||
Using Domain | Yes | datatype alias for more intuitive and consistent representation |
Defining mandatory or initial data | Yes | |
Generate DDL | Yes | - Oracle Database 9i/10g/11g/12c, SQL Server 2000/2005/2008/2012, DB2/UDB 7.1/8.1/9 - Not supporting MySQL and PostgreSQL as of 4.2 |
Using name template | Yes | PK, FK, UK, Check, Index, ... |
- Readings
- How to export image of relational model in Oracle SQL Data Modeler? (Oct 9 '13)
[File > Print Diagram > To Image File]
- Configuring Display of Model Relationships in Oracle SQL Developer Data Modeler (January 31, 2013)
- font, color, ...
- How to export image of relational model in Oracle SQL Data Modeler? (Oct 9 '13)
SQL Power Architect
- http://www.sqlpower.ca/page/architect
- Desc. : Data Modeling & Profiling Tool
- License :
- Features supported or not supported
Feature | Supporting | Remarks |
---|---|---|
Defining AK(Alternative Key) | Yes | |
Defining View | No | |
Defining FK from AK | No |
UMLGraph
- http://www.umlgraph.org/
- Desc. : allows the declarative specification and drawing of UML class and sequence diagrams.
- License : BSD license
- Readings
SQL Client/Database Management
SQuirreL SQL Client
- http://www.squirrelsql.org/
- Desc. : A graphical Java program that will allow you to view the structure of a JDBC compliant database, browse the data in tables, issue SQL commands etc.
IBM Data Studio
- http://www-01.ibm.com/software/data/optim/data-studio/
- Desc. : Provides database developers and DBAs an integrated, modular environment for productive administration of DB2 for Linux, UNIX and Windows.
Oracle SQL Developer
SchemaSpy
- http://schemaspy.sourceforge.net/
- Desc. : a Java-based tool (requires Java 5 or higher) that analyzes the metadata of a schema in a database and generates a visual representation of it in a browser-displayable format.
- License : Lesser GNU Public License 2.1
SQLLine
- http://sqlline.sourceforge.net/
- Desc. : a pure-Java console based utility for connecting to relational databases and executing SQL commands.
Robomongo
- http://robomongo.org/
- Desc. : Shell-centric cross-platform MongoDB management tool
HeidiSQL
- http://www.heidisql.com/
- Desc. : a useful and reliable tool designed for web developers using the popular MySQL server, Microsoft SQL databases and PostgreSQL
- License : GPL
Software License Analysis
FOSSology
- http://www.fossology.org/projects/fossology
- Desc. : scanning files for licenses and copyrights.
- License : GPL version 2
Ninka
- http://ninka.turingmachine.org/
- Desc. : a lightweight license identification tool for source code
- License : Affero General Public License version 3.0
OSS Discovery
- http://ossdiscovery.sourceforge.net/
- Desc. : finds the open source software embedded in applications and installed on computers.
- License : GNU Affero Public License v3
System Monitoring
nmon
- http://www.ibm.com/developerworks/wikis/display/WikiPtype/nmon
- Desc. : free tool giving you a huge amount of important performance information in one go.
top
- http://www.unixtop.org/
- Desc. : a UNIX utility that provides a rolling display of top cpu using processes.
- References
- Readings
- How To Use The Linux Top Command To Show Running Processes (February 08, 2017)
- How to use the Linux top command(June 3 2016)
- Why Process CPU % Usage larger than Total CPU Time(Jun 27 '11)
- Tips
- Remarkable commands
W
: Write-the-Configuration-Filef | F
: :Fields-ManagementH
: Threads-mode toggleI
: Irix/Solaris-Mode togglet
: Task/Cpu-States togglem
: Memory/Swap-Usage toggle1
: Single/Separate-Cpu-States togglec
: Command-Line/Program-Name toggleu | U
: Show-Specific-User-Onlyi
: Idle-Process toggle
- Remarkable commands
htop
- http://hisham.hm/htop/
- Desc. : an interactive process viewer
- License : General Public License, version 2 (GPL-2.0)
- Written in : C
- Sources : https://github.com/hishamhm/htop
Nmap
- http://nmap.org/
- Desc. : a free and open source (license) utility for network discovery and security auditing.
Wireshark
- http://www.wireshark.org/
- Desc. : lets you capture and interactively browse the traffic running on a computer network.
- References
- Readings
- How To Set Up a Capture
- Loopback capture setup
- Wireshark basics 101: A simple concise tutorial for beginners (August 17, 2013)
- How to Use Wireshark to Capture, Filter and Inspect Packets
- Wireshark: A Guide to Color My Packets (1st July 2014)
- Getting Started with Wireshark (11, 07, 2014)
- Let me tell you about Wireshark 2.0 (November 6, 2015)
- Typical display filters
ip.src == 192.168.1.31 and ip.addr == 203.252.150.28 and http
- Typical capture filters
iperf
- http://software.es.net/iperf/
- Desc. : a tool for active measurements of the maximum achievable bandwidth on IP networks.
- License : BSD
- Readings
Process Explorer
- https://technet.microsoft.com/en-us/library/bb896653.aspx
- Desc. : shows you information about which handles and DLLs processes have opened or loaded.
Process Monitor
- https://technet.microsoft.com/en-us/library/bb896645.aspx
- Desc. : an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity.
DebugView
- https://technet.microsoft.com/en-us/sysinternals/debugview.aspx
- Desc. : an application that lets you monitor debug output on your local system, or any computer on the network that you can reach via TCP/IP.
Testing
JMeter
- http://jmeter.apache.org/
- Des. : a 100% pure Java application designed to load test functional behavior and measure performance.
- License : Apache License v2
- References
- Readings
- Free Jmeter Tutorial
- Using User Defined Variables
- Use request value from list of values in JMeter (Jun 14 '11)
- JMeter tips (JUL 11, 2005)
- Solved “java.net.BindException: Address already in use: connect” issue on Windows(November 23, 2011)
- JMeter HTTP Authorization Manager
- Here’s What to Do to Combine Multiple JMeter Variables (May. 24th, 2016)
- HOW TO ADD THINK TIME BETWEEN REQUESTS IN JMETER (October 9, 2014)
- A Comprehensive Guide to Using JMeter Timers (Jul. 5th, 2015)
- Companions
Tips and Trics
Using variables in assertion
In 'Response Assertion', the variable defined via 'CSV Data Set Config' or 'User Defined Variables' can be used in test pattern.
In the following sample, certHash
is variable defined in 'CSV Data Set Config'
${__escapeOroRegexpChars(\"cert_hash\":\"${certHash}\")}
Eclipse TPTP
- http://www.eclipse.org/tptp/
- Desc. : provides an open platform supplying powerful frameworks and services that allow software developers to build unique test and performance tools, both open source and commercial, that can be easily integrated with the platform and with other tools.
Testopia
- http://www.mozilla.org/projects/testopia/
- Desc. : A test case management extension for Bugzilla.
TestLink
- http://sourceforge.net/projects/testlink/
- Desc. : a web based Test Management tool.
- License : GPLv2
FitNesse
- http://fitnesse.org/
- Desc. : The fully integrated standalone wiki and acceptance testing framework.
CodePro Analytix
- https://developers.google.com/java-dev-tools/codepro/doc/
- Desc. : the premier Java software testing tool for Eclipse developers who are concerned about improving software quality and reducing developments costs and schedules.
- License :
Postman
- https://www.getpostman.com/
- Desc. : Build, test, and document your APIs faster.
- Readings
Phoronix Test Suite
- http://www.phoronix-test-suite.com/
- Desc. : the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems.
- License : GNU GPL
- Readings
Blogbench
- https://www.pureftpd.org/project/blogbench
- Desc. : a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server.
Log Viewer
Lilith
- http://lilith.huxhorn.de/
- Desc. : Lilith is a logging and access event viewer for the Logback logging framework, log4j and java.util.logging.
- License :
Documentation
Swagger
- http://swagger.io/
- Desc. : a powerful open source framework backed by a large ecosystem of tools that helps you design, build, document, and consume your RESTful APIs
- License :
- Sources : https://github.com/swagger-api
- References
- OpenAPI Specification Ver. 2.0 (fka Swagger RESTful API Documentation Specification)
- OpenAPI Spec Schema 2.0
- Swagger-Core Annotations
- Swagger-Core Annotations API (Javadoc)
- Swagger Code Generator
- allows generation of API client libraries (SDK generation), server stubs and documentation automatically given an OpenAPI Spec
- Swagger Code Generators
SwaggerGenerator
,StaticHtml2Generator
,JMeterCodegen
, ...
- Readings
- Writing OpenAPI (Swagger) Specification Tutorial
- How to split a Swagger spec into smaller files (Jul 16, 2015)
- Generating Swagger spec and enabling swagger-ui for REST based applications (Jun 2, 2016)
- Using Javadocs to generate Swagger document
- Spring Rest API with Swagger – Fine-tuning exposed documentation (23.11.2014)
Swagger2Markup
- https://github.com/Swagger2Markup/swagger2markup
- Desc. : simplify the generation of an up-to-date RESTful API documentation by combining documentation that’s been hand-written with auto-generated API documentation produced by Swagger.
- License : Apache License Version 2.0
- Readings
- Swagger2Markup 1.3.1 Documentation
- Create HTML documentation from Swagger via Maven (Jún 29, 2016)
- using both Swagger2Markup Maven Plugin and Asciidoctor Maven Plugin
- Asciidoctor Maven Plugin
- Having fun with REST API documentation (Sep 22, 2015)
- using both Swagger2Markup Maven Plugin and Asciidoctor Maven Plugin
org.asciidoctor.Attributes
source
- Swagger2Markup Maven template project
SpringFox
- https://springfox.github.io/springfox/
- Desc. : Automated JSON API documentation for API's built with Spring
- License :
- Sources : https://github.com/springfox/springfox
- Readings
- Samples
Enunciate
- http://enunciate.webcohesion.com/
- Desc. : an engine for enhancing your Java Web service API
Doxygen
- http://www.stack.nl/~dimitri/doxygen/
- Desc. : a documentation system for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors), Fortran, VHDL, PHP, C#, and to some extent D.
- License : GPL
- Readings
Sphinx
- http://sphinx-doc.org/
- Desc. :Python documentation generator
- License : BSD license
Publican
- https://fedorahosted.org/publican/
- Desc. :A tool for publishing material authored in DocBook XML.
Pandoc
- http://johnmacfarlane.net/pandoc/
- Desc. : convert documents in markdown, reStructuredText, textile, HTML, DocBook, or LaTeX to HTML formats, word processor formats, documentation formats, TeX formats, PDF, and/or lightweight markup formats.
- License : GPL
- Sources
- Readings
jax-doclets
- http://fromage.github.io/jax-doclets/
- Desc. : contains doclets for modern Java APIs such as JAXB, JAX-RS and JPA.
- License : LGPL
- Maven artifacts : com.lunatech.jax-doclets » doclets
- Sources : https://github.com/FroMage/jax-doclets
- Readings
xs3p
- http://sourceforge.net/projects/xs3p/
- Desc. : an XSLT stylesheet that will generate an XHTML document from an XSD schema.
- License : DSTC Public License (DPL)
- Maven artifacts
- Sources
- Readings
Enunciate
- http://enunciate.codehaus.org/
- Desc. : an engine for dramatically enhancing your Java Web service API.
- License : Apache License Version 2.0
MireDot
- http://www.miredot.com/
- Desc. : REST API documentation generator for Java that generates beautiful interactive documentation with minimal configuration effort.
- License : complex -> http://www.miredot.com/price/
Text Editor
vi
- Readings
- Basic vi Commands
- Vi Cheat Sheet
- vi Manual by Tony Chen
- Split screen in vi (February 17th, 2009)
jEdit
- http://www.jedit.org/
- Desc. : a mature programmer's text editor with hundreds (counting the time developing plugins) of person-years of development behind it
- License : GPL 2.0
Atom
- https://atom.io/
- Desc. : a text editor that's modern, approachable, yet hackable to the core
- License : MIT
- Sources : https://github.com/atom/atom
- Readings
Build
Maven
- http://maven.apache.org/
- Desc. : a software project management and comprehension tool.
- On Maven
Ant
- http://ant.apache.org/
- Desc. : a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other.
- On Ant
Gradle
- http://www.gradle.org/
- Desc. : automate the building, testing, publishing, deployment and more of software packages or other types of projects such as generated static websites, generated documentation or indeed anything else.
- License :
- Source :
Referencs
- Common commands
Command | Description | Reamrks |
---|---|---|
$ gradle tasks --all |
gives you a list of the main tasks of the selected project |
Readings
- Aggregate Javadoc in multiproject Gradle builds (February 25, 2016)
Package Management
- Package Management Softwares
Software | Scope | Remarks |
---|---|---|
APT (Advanced Package Tool) | Ubuntu, Debian | a set of tools for managing Debian packages, and therefore the applications installed on your Debian system |
RPM | CentOS, RHLP | a powerful command line driven package management system capable of installing, uninstalling, verifying, querying, and updating computer software packages |
yum | CentOS, RHLP | an automatic updater and package installer/remover for rpm systems |
Chocolatey | Windows | the package manager for Windows |
npm | JavaScript | the package manager for JavaScript |
yarn | JavaScript | a package manager for JavaScript |
pip | Python | the PyPA recommended tool for installing Python packages |
RubyGems | Ruby | allows you to easily download, install, and use ruby software packages on your system |
Bundler | Ruby | provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed |
RPM
- http://rpm.org/
- Desc. : a powerful command line driven package management system capable of installing, uninstalling, verifying, querying, and updating computer software packages.
- Readings
- Example
$ rpm -Uhv mysql-workbench-community-6.2.3-1.el6.x86_64.rpm
//install or upgrade. (do NOT use rpm -i ... as possible)
$ rpm -ql mysql-workbench-community
//list files in a installed package
$ rpm -qlp mysql-workbench-community-6.2.3-1.el6.x86_64.rpm
//list files in a rpm file
$ rpm -qp --scripts mysql-workbench-community-6.2.3-1.el6.x86_64.rpm
//list install/erase scriptlets in a rpm file
yum
- http://yum.baseurl.org/
- Desc. : an automatic updater and package installer/remover for rpm systems.
- Readings
- Fedora Yum Guide
- Managing Software with
yum
- Managing Packages With Yum
yum list, yum install, yum remove, yum update, yum check-update, yum info, yum search, yum provides, yum clean
- Basic Yum Commands and how to use them
- Yum groups and repositories
yum grouplist, yum groupinfo, yum groupinstall, yum groupremove, yum groupupdate
- How To Download a RPM Package Using yum Command Without Installing On Linux (JULY 9, 2008)
- Listing Files Contained in a Package
- CentOS / RHEL: List All Configured Repositories (FEBRUARY 25, 2013)
- Adding, Enabling, and Disabling a Yum Repository
- Using RPMforge
- Yum install maven : Yes you can ! (August 21, 2013)
- Commands
- Examples
yum -v repolist enabled
//list only enabled repositories
yum -v list all subversion --enablerepo=rpmforge-extras
//search subversion packages both installed and available including 'rpmforge-extras' repository
yum -v list installed
//list all installed packages
APT
- https://wiki.debian.org/Apt
- Desc. : a free software user interface that works with core libraries to handle the installation and removal of software on the Debian, Slackware and other Linux distributions
- Readings
RubyGems
- http://rubygems.org/
- Desc. : allows you to easily download, install, and use ruby software packages on your system
- License : Ruby License
- Written in : Ruby
- Readings
- RubyGems at Wikipedia
- RubuGems official guide
- RubyGems Basics
- Command Reference
gem install
,gem list
Bundler
- http://bundler.io/
- Desc. : provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed
- License : MIT License
- Sources : https://github.com/bundler/bundler
- Readings
- Bundler official documentation
Gemfile
: A format for describing gem dependencies for Ruby programs
Frontend
MySchedule
- http://code.google.com/p/myschedule/
- Desc. : a web application for managing Quartz Schedulers.
Networking
OpenSSH
- http://www.openssh.org/
- Desc. : a FREE version of the SSH connectivity tools that technical users of the Internet rely on.
- Readings
- Readings
OpenSSL
- https://www.openssl.org/
- Desc. : a robust, commercial-grade, full-featured, and Open Source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols as well as a full-strength general purpose cryptography library.
- License :
- Readings
curl
- http://curl.haxx.se/
- Desc. : a command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP.
- License :
- Readings
PuTTY
- http://www.putty.org/
- Desc. : an SSH and telnet client, developed originally by Simon Tatham for the Windows platform.
- Readings
- PuTTY 0.68 User Manual
- How to make PuTTY settings persistent? (May 26 '09)
- Putty title changes after login (Nov 26 '12)
- [Terminal] -> [Features] -> [Disable remote-controlled window title changing]
- puTTY SSH connections dropping when on wifi instead of ethernet
- [Connection] -> [Enable TCP keepalives (SO_KEEPALIVE option)]
Companions
PuTTY Session Manager
- https://puttysm.sourceforge.io/
- Desc. : a tool that allows system adminstrators to organise their PuTTY sessions into folders and assign hotkeys to their favourite sessions.
ConEmu
- http://conemu.github.io/
- Desc. : a Windows console emulator with tabs, which presents multiple consoles and simple GUI applications as one customizable GUI window with various features.
Pageant
Poderosa
- http://poderosa.sourceforge.net/
- Desc. : a terminal emulator for .NET Framework environment
- Sources : https://github.com/poderosaproject/poderosa
- License : Apache License 2.0
Graphics
Graphviz
- http://www.graphviz.org/
- Desc. : open source graph visualization software.
- License : EPL
Mscgen
- http://www.mcternan.me.uk/mscgen/
- Desc. : a small program that parses Message Sequence Chart descriptions and produces PNG, SVG, EPS or server side image maps (ismaps) as the output.
- License : GPLv2
OpenSG
- http://www.opensg.org/
- Desc. : a portable scenegraph system to create realtime graphics programs.
- License : LGPL
OpenSceneGraph
- http://www.openscenegraph.org/projects/osg
- Desc. : an open source high performance 3D graphics toolkit, used by application developers in fields such as visual simulation, games, virtual reality, scientific visualization and modelling. Written entirely in Standard C++ and OpenGL.
- License : LGPL
Visualization
XsdVi
- http://sourceforge.net/projects/xsdvi/
- Desc. : transform W3C XML Schema instances into interactive diagrams in SVG format.
- License : GPLv2
Data Analysis
Cube
- http://square.github.com/cube/
- Desc. : a system for collecting timestamped events and deriving metrics.
- License : Apache License 2.0
ETL
Scriptella
- http://scriptella.javaforge.com/
- Desc. : An open source ETL (Extract-Transform-Load) and script execution tool written in Java.
Servers or Engines
Hypervisor
VirtualBox
- https://www.virtualbox.org/
- Desc : a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use.
- License : GPL version 2
- References
- User Manual
VBoxManage
reference : VirtualBox CLI- Virtual networking
- Readings
- Install Ubuntu on Oracle VirtualBox
- CentOS as a Guest OS in VirtualBox
- Install a CentOS 7 Minimal Virtual Machine with VirtualBox
- Using a raw host hard disk from a guest
- Four virtual disk formats every VDI admin needs to know
- How to create and start VirtualBox VM without GUI (May 9, 2013)
- Accessing your Virtualbox Guest from your Host OS : Host-only networks
Xen Project Hypervisor
- https://xenproject.org/
- Desc. : an open-source type-1 or baremetal hypervisor
- License :
- Sources
- Readings
DBMS
HSQLDB
- http://hsqldb.org/
- Desc. : the leading SQL relational database software written in Java.
- License : BSD License
- Maven artifacts
- hsqldb » hsqldb : 1.6.x ~ 1.8.x
- org.hsqldb : 2.0.x ~
- Source : http://sourceforge.net/p/hsqldb/svn/HEAD/tree/
- References
- HyperSQL 2.x User Guide
- HyperSQL 1.8 User Guide
- HyperSQL 2.x API
org.hsqldb.server.Server
API- shows command line options and format of
server.properties
file
- shows command line options and format of
- Readings
Apache Derby
- http://db.apache.org/derby/
- Desc. : an open source relational database implemented entirely in Java
- License : Apache License, Version 2.0
- Readings
J2EE Application Server
Jetty
- http://jetty.codehaus.org/
- Desc : provides an HTTP server, HTTP client, and javax.servlet container.
- License : http://www.eclipse.org/jetty/licenses.php
- Source
- Binaries
- Documentations
- Jetty Versions
- Jetty 9
- Jetty 7/8
- Jetty 6
- Articles
- Jetty 9
- Embedded Examples/Web Application with JSP
- Example: Embedded Jetty w/ JSP Support
- Using Spring to Configure Jetty
<bean id="server" name="Main" class="org.eclipse.jetty.server.Server" init-method="start" destroy-method="stop">
- Jetty Logging
- JMX
- How do you get embedded Jetty 9 to successfully resolve the JSTL URI?(Jul 16 '13)
- Jetty 7/8
- How to make Jetty dynamically load "static" pages
- Debugging with the Maven Jetty Plugin inside Eclipse
- Avoiding Jetty Locking Static Files(January 10, 2012)
- Jetty Cross Origin Filter (Nov 28 '11)
- Jetty 9
Tomcat
- http://tomcat.apache.org/
- Desc : an open source software implementation of the Java Servlet and JavaServer Pages technologies.
- License : Apache License version 2
- References
JBoss AS
- References
- Articles
- fundamental
- Startup parameter
- Enabling remote access to JBoss 4.2
- Class loading configuration
- Explains how-to setup class loading isolation of J2EE modules using <loader-repository> element in
jboss-app.xml
,jboss.xml
orjboss-web
file.
- Explains how-to setup class loading isolation of J2EE modules using <loader-repository> element in
- Configuring the Deployment Scanner in conf/jboss-service.xml
- JBoss deployment directory configuration
- Adding a custom deploy folder
- data access
- security
- networking
- programming
- fundamental
WildFly
- References
- Wildfly 10.0.0.Final Model Reference
- WildFly 10 Documentation
- WildFly Quick Starts : provide small, specific, working examples that can be used as a reference for your own project
- WildFly 10 CLI Recipes
- Command line parameters
- command-line parameters of
$JBOSS_HOME/bin/domain.sh
and$JBOSS_HOME/bin/standalone.sh
- command-line parameters of
- Readings
- WildFly 10 Deployment Scanner configuration
- Useful JBoss and WildFly Command Line Parameters, System Properties and Environment Variables (Jan 31st, 2015)
- Adding WildFly User -
add-user.sh
- Connecting VisualVM or JConsole to Wildfly 10 (2016. 5. 20)
service:jmx:http-remoting-jmx://hostname:9990
- Monitoring WildFly using VisualVM (04 July 2014)
Undertow
- http://undertow.io/
- Desc. : a flexible performant web server written in java, providing both blocking and non-blocking API’s based on NIO.
- Written in : Java
- License :
- Readings
TomEE
- http://openejb.apache.org/
- Desc : an all-Apache Java EE 6 Web Profile certified stack where Tomcat is top dog.
OpenEJB
- http://openejb.codehaus.org/
- Desc : an open source, modular, configurable, and extendable EJB Container System and EJB Server.
- License :
Application Server
Apache HTTP Server
- References
- Apache 2.4 Core Features
- Directives
- Apache Authentication via HTTP Basic or Digest
- How can I check which version of apache is installed on a Debian machine? (Nov 14 '08)
- How do I find the version of Apache running without access to the command line? (Oct 3 '08)
- Apache2: 'AH01630: client denied by server configuration' (Aug 23 '13)
- How To Configure Logging And Log Rotation In Apache On An Ubuntu VPS (August 19, 2013)
Nginx
- http://nginx.org/
- Desc. : a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server
- License :
- Written in : C
- Readings
- Nginx passes Apache as Web server of choice among top sites(May 27, 2014)
- Nginx overtakes Microsoft as No. 2 Web server(Jan 4, 2012)
mongoose
- https://github.com/cesanta/mongoose
- Desc. : Embedded web server for C/C++
- License : GPL v2
ZooKeeper
- http://zookeeper.apache.org/
- Desc. : an open-source server which enables highly reliable distributed coordination.
- License : Apache License
- Written in : Java
- Source
- master : https://github.com/apache/zookeeper
- release 3.3.6 : https://github.com/apache/zookeeper/tree/release-3.3.6
- Maven artifacts :
org.apache.zookeeper » zookeeper
- References
- Readings
- ZooKeeper JMX
- Running Zooinspector
- How to Run ZooInspector(31.5.13)
- Using Maven
- Install and start Zookeeper server on Windows(August 12, 2014)
Media Streaming Server
LIVE555 Media Server
- http://www.live555.com/mediaServer/
- Desc. : http://www.live555.com/mediaServer/
- License : LGPL
- Sources :
Red5
- https://github.com/Red5
- Desc. : an Open Source Flash Server written in Java
- License : Apache License 2.0
- Sources : https://github.com/Red5
- Maven artifacts : org.red5
- References
- Packages and Classes
Constants
- includes constants for RTMP packet types and shared objects
RTMPClient extends BaseRTMPClientHandler
BaseRTMPClientHandler extends BaseRTMPHandler implements IRTMPClient
BaseRTMPHandler implements IRTMPHandler, Constants, StatusCodes
RTMPConnection
Channel
org.red5.server.jmx.mxbeans
packages- Events
Aggreate.java
source- Aggreation of video data and audio data
- Built-in Streams
AbstractClientStream
ClientBroadcastStream.java
- Mandatory sample for real-world stream implementation
- Note on
ClientBroadcastStreamMXBean
,getCodecInfo()
,metaData
and call ofVideoCodecFactory.getVideoCodec
.
- Readings
- How to format Adobe Flash RTMP URLs(10-31-2014)
- How to record RTMP flash video streams using Red5(2009-04-04)
- How to extract snapsot image from RTMP stream’s videodata in Red5 App
- Using RTMP Streaming
- RTMP protocol and extract RTMP video streams H264 video files(2014-02-02)
- Writing a Red5 Java Web App to Handle RTMP Streams(2012-12-04)
- Extract frames as images from an RTMP stream in real-time(Feb 2 '14)
Darwin Streaming Server
- http://dss.macosforge.org/
- Desc. : the open source server technology that allows you to send streaming media to clients across the Internet using the industry standard RTP and RTSP protocols.
- License : Apple Public Source License
NoSQL
- Classification of NoSQL
- Distributed cache
- Java Caches: Ehcache, Hazelcast, Infinispan
- A comparative study of distributed caches (August 28, 2015)
- Memcached, Redis, Hazelcast
- Comparing Distributed Caching Frameworks (Apr 6, 2015)
- Hazelcast, Memcached, Redis
- From distributed caches to in-memory data grids (May 20, 2012)
Redis
- http://redis.io/
- Desc : an open-source, networked, in-memory, key-value data store with optional durability.
- License : BSD
- Readings
- Commands
- An introduction to Redis data types and abstractions
- Data types
- Redis configuration
- The Little Redis Book
- Example of storing nested data structures in redis
- How to access Redis log file (May 2 '13)
- Redis-server - FATAL CONFIG FILE ERROR - Filepath Space problem(May 18 '11)
- Use "
C:/Program Files (x86)/My Application/log/redis.log
" instead of "C:\Program Files (x86)\My Application\log\redis.log
"
- Use "
Companions
lettuce
- https://github.com/mp911de/lettuce
- Desc. : Advanced Redis client for thread-safe sync, async, and reactive usage.
- License : Apache License 2.0
- Readings
Spring Data Redis
- http://projects.spring.io/spring-data-redis/
- Desc. : provides easy configuration and access to Redis from Spring applications.
embedded-redis
- https://github.com/kstyrc/embedded-redis
- Desc. : Redis embedded server for Java integration testing
- Licesne : Apache License, Version 2.0
- Readings
FastoRedis
- http://fastoredis.com/
- Desc. : a cross-platform open source Redis management tool (i.e. Admin GUI)
- License : GPL
- Readings
MongoDB
- https://www.mongodb.org/
- Desc. : an open-source, document database designed for ease of development and scaling
- License : Apache License Version 2.0
- Sources : https://github.com/mongodb/mongo
CouchDB
- http://couchdb.apache.org/
- Desc. : open source database software that focuses on ease of use and having an architecture that "completely embraces the Web"
- License :
- Sources : https://github.com/apache/couchdb
- References
- Readings
- Erlang
inet
module > socket optionshttpd
/socket_options
- MochiWeb startup options
httpd
/server_options
name, ip, backlog, nodelay, acceptor_pool_size, ssl, profile_fun, link, recbuf
- CouchDB JIRA > Keep Alive connections with zero size attachments
- Erlang
Version Control Software
Subversion
- Download
- References
- Version Control with Subversion 1.8 (draft)
- Version Control with Subversion 1.7
- Compatibility concerns
- Older clients and servers interoperate transparently with 1.7 servers and clients.
- Existing working copies created with Subversion 1.6 and earlier need to be upgraded before they can be used with a Subversion 1.7 client.
- Working Copy Metadata Storage Improvements
- Traversing Branches
- Compatibility concerns
- Version Control with Subversion 1.6
- Version Control with Subversion 1.5
- Version Control with Subversion 1.4
- Subversion runtime configuration
- Readings
- Using negative patterns for Subversion's svn:ignore property
- Set svn:ignore for multiple files from command line
- Migrating Subversion Repository Data Elsewhere
- Integrating Redmine with Subversion(October 1, 2012)
- Best way to link Redmine issue to SVN revision
- client configuration of Subvesion
- The location of client configuration files for subversion is dependent on operating system.
- Windows :
%USERPROFILE%\Application Data\Subversion\config
- Linux :
~/.subversion/config
- Windows :
- Configurable options
- The location of client configuration files for subversion is dependent on operating system.
- Per repository AuthzSVNAccessFile apache rule with a single apache config file or other shell script solution (Jan 7 2014)
- Add all unversioned files to SVN(Oct 21 '09)
- What is the correct way to restore a deleted file from SVN?(Jan 29 '09)
- How do I tell Subversion to treat a file as a binary file?(Sep 16 '08)
svn:mime-type = application/octet-stream
- How does Subversion handle binary files?
svn checkout
svn diff
svn update
- How do I avoid “svn: Out of Date:” problems? (May 2 '09)
- I'm trying to commit, but Subversion says my working copy is out of date ?
- Is it possible to always (force) overwrite local changes when updating from SVN? Ignore conflicts?(Sep 14 '10)
- update ignoring(overwriting) local change =
svn revert
+svn update
- update ignoring(overwriting) local change =
svn delete
- How do you remove Subversion control for a folder?(Sep 30 '08)
- Remarks on '
svn delete --keep-local
' and single.svn
directory per working copy feature of Subversion 1.7.
- Remarks on '
- How do you remove Subversion control for a folder?(Sep 30 '08)
- Examples
$ svn info . //show status including working copy path, repository path, base revision and et al.
$ svn status . //show only locally modified items under the current directory of working copy
$ svn update //update working copy
$ svn commit //send changes from your working copy to the repository
$ svn --force --depth infinity add . //add all unversioned files to local change
$ svn copy ^/trunk/foundation ^/branches/foundation-20141123 -c "Created a new branch of /trunk/foundation" //making branch
Git
References
command | description | remark |
---|---|---|
git-clone |
||
git-status |
||
git-add |
||
git-commit |
||
git-push |
||
git-pull |
||
git-branch |
List, create, or delete branches | |
git-tag |
Create, list, delete or verify a tag object signed with GPG | |
git-remote |
Manage set of tracked repositories | |
git-checkout |
||
gitignore |
($HOME/.config/git/ignore, $GIT_DIR/info/exclude, .gitignore )
| |
git-diff |
||
git-rebase |
Reapply commits on top of another base tip | |
git-reset --hard |
Resets the index and working tree. Any changes to tracked files in the working tree since <commit> are discarded. | |
git-clean |
Remove untracked files from the working tree |
- Git Reference
- Pro Git
- GitHub Bootcamp
- As of Nov. 1st 2012, one important thing is missing. That is sharing the project in Eclipse, after cloning the fork in the server. Maybe it could be a defect of Eclipse, not a miss of the article.
- Git Cheatsheet
- Git Tutorials
- Git Workflows
- EGit User Guide
Articles
- What's a “fast-forward” in Git?
- Is there any way to clone a git repository's sub-directory only?
- How to change file premissions in Git on Windows (Nov. 2011)
- Git - Get Current Working Copy Version (Jan 23 '12)
git log -1
- How to get latest git commit hash command (Mar 28 '13)
- Dealing with line endings
- What does tree-ish mean in Git?(Oct 28 '10)
- Ignoring files
- gitignore: Ignore all files in folder hierarchy except one specific filetype(
- Git v2.0.0, what changed, and why should you care (2014/05/29)
- Adding an existing project to GitHub using the command line
- Revision
- Status
- Branch
- Checkout
- How to checkout remote git tag (Mar 14 '16)
git checkout v1.0.0, git checkout tags/v1.0.0
- How to get back to most recent version in Git? (Aug 24 '10)
git checkout master
- How to checkout remote git tag (Mar 14 '16)
- Subtree
- About Git subtree merges
- to manage multiple projects within a single repository
- Mastering Git subtrees
- About Git subtree merges
- Diff
Tools
Typical Usage
Adding new items, committing and pushing them
$ git pull origin
$ git status . //identifies added, modified or untracked items under current local directory
$ git add . //updates index
$ git status . //confirms the changes to be committed
$ git commit -m "message ..." //commits changes into local repository
$ git status . //confirms all changes are committed
$ git push //pushes changes in local repository onto the remote repository
Moving working tree to a specific branch or tag
$ git branch //lists branches
$ git tag //lists tags
$ git checkout release-1.1 //updates working tree to release-1.1
Working with remote repositories
$ git remote -v //lists tracking repositories
$ git branch -r //lists branches in tracking repositories
Working with branches or tags
$ git checkout -b gh-pages origin/gh-pages //copy remote branch to local repository
Removing uncommitted changes on tracked files in the working tree
$ git reset --hard //Resets the index and working tree. Any changes to tracked files in the working tree since <commit> are discarded.
Removing untracked files and directories in the working tree
$ git clean -ndx // -n: dry run, -d: directory also, -x: skip gitignore
...
$ git clean -ffdx // -ff : force including files manged by different Git repository
...
For more, refer
- how to remove untracked files in Git? (Nov 20 '11)
- How to remove local (untracked) files from the current Git working tree? (Sep 14 '08)
GitHub
GitHub Pages
- Get Started With GitHub Pages (Plus Bonus Jekyll)(18 December 2013)
- A blog site with Jekyll, Github Pages Eclipse Che and Codenvy
- Using Jekyll as a static site generator with GitHub Pages
- GitHub Pages themes
- primer, cayman, hacker, minimal, midnight
EGit
- http://www.eclipse.org/egit/
- Desc : an Eclipse Team provider for the Git version control system.
- License
- Readings
- EGit User Guide
- Git For Eclipse Users
- EGit Getting Started
- EGit/GitHub/User Guide : explains how to use EGit Mylyn GitHub connector.
- EGit Compare shows all lines as changed (Dec 30 '13)
- EGit doesn't support
.gitattributes
file.
- EGit doesn't support
Harvest
Gitorious
- Desc : Git hosting and collaboration software that you can install yourself
- License : GNU Affero General Public License
GitLab
- https://github.com/gitlabhq/gitlabhq
- Desc : self hosted Git management software
- License : MIT license
Team Foundation Version Control
- https://msdn.microsoft.com/en-us/library/ms181237.aspx
- Desc. : a centralized version control system.
- Readings
- Customize which files are ignored by version control
.tfignore
, recursive by default
- Customize which files are ignored by version control
Repository Management Software
Artifactory
- http://sourceforge.net/projects/artifactory/
- Desc : Binary Repository Manager for Maven, Ivy, Gradle modules, etc.
- License : LGPLv3
Configuration Management Software
- Infrastructure as code (IaC)
- Comparison of open-source configuration management software
- Deployment Management Tools: Chef vs. Puppet vs. Ansible vs. SaltStack vs. Fabric (August 5, 2015)
Chef
- https://www.chef.io/
- Desc. : a powerful automation platform that transforms infrastructure into code
- License : Apache License 2.0
- References
Puppet
- https://puppet.com/
- Desc. :
- License :
- References
Ansible
- https://www.ansible.com/
- Desc. : can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.
- License :
- Written in :
- Sources : https://github.com/ansible/ansible
References
- Ansible 2.4
- Ansible 2.4 Documentation
- Inventory : Hosts and Groups
- Configuration File :
ansible.cfg
- Module Index
- Return Values
- Variables
- Magic Variables(Built-in Variables)
hostvars, group_names, groups, ansible_play_hosts, ansible_play_batch, inventory_dir, inventory_file, playbook_dir, ansible_check_mode
- Fact Caching
- Magic Variables(Built-in Variables)
- Playbooks
- Directives Glossary
- Loops
- Filters
- Lookups
- Vault
- Ansible Container
- Jinja2 2.10
- Template Designer Documentation
- inline
if
expressions - Jinja2
selectattr( )
Filterselectattr('email', 'undefined'), selectattr("type", "equalto", "floating")
Concepts
playbook = plays+
play = (host+, task+)
task = (name, module, ...)
Modules
Module | Description | Remarks |
---|---|---|
debug
|
Print statements during execution | |
assert
|
Asserts given expressions are true | |
fail
|
Fail with custom message | |
pause
|
Pause playbook execution | |
copy
|
Copies files to remote locations | |
file
|
Sets attributes of files | |
stat
|
Retrieve file or file system status | |
template
|
Templates a file out to a remote server | |
shell
|
Execute commands in nodes | /bin/sh , /bin/bash
|
command
|
Executes a command on a remote node | |
expect
|
Executes a command and responds to prompts | |
get_url
|
Downloads files from HTTP, HTTPS, or FTP to node | curl, wget |
uri
|
Interacts with HTTP and HTTPS web services and supports Digest, Basic and WSSE HTTP authentication mechanisms | curl, weg |
apt
|
Manages apt-packages | |
pip
|
Manages Python library dependencies | |
docker_container
|
Manage the life cycle of docker containers | |
docker_image
|
Manage docker images | |
git
|
Deploy software (or files) from git checkouts |
Facts
Category | Fact | Description | Values | Remarks |
---|---|---|---|---|
Common | ansible_architecture
|
"x86_64" | ||
ansible_distribution
|
"Ubuntu" | |||
ansible_distribution_version
|
"16.04" | |||
ansible_distribution_major_version
|
"16" | |||
ansible_distribution_release
|
"xenial" | |||
ansible_hostname
|
hostname | |||
ansible_env
|
ansible_env.HOME, ansible_env.PATH, ansible_env.PWD
| |||
ansible_interfaces
|
["lo", "eth0", "eth1", "eth2", "eth3", "docker0", ...] | Array | ||
ansible_os_family
|
"Debian" | |||
ansible_pkg_mgr
|
"apt" | |||
ansible_check_mode
|
false
|
|||
inventory_hostname
|
name of the host as configured in Ansible's inventory host file (usually hosts.yml )
|
"m001", "m002" | not always same with ansible_hostname
|
Filters and Tests
Filter | Description | Applied to | Sample | Remarks |
---|---|---|---|---|
regex_replace
|
replace text in a string with regex | string | common.quorum.download.constellation|regex_replace('^.*/', '')
|
Regex in Python 3 |
regex_search
|
search a string with a regex | string | ||
select
|
Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. | a sequence of objects | ||
selectattr
|
Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. | a sequence of objects | ||
reject
|
Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. | a sequence of objects | ||
rejectattr
|
Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. | a sequence of objects | ||
map
|
Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. | a sequence of objects | ||
attr
|
Get an attribute of an object. | an object | ||
list
|
Convert the value into a list. |
Roles
Role | Location | Remarks |
---|---|---|
apparmor | https://github.com/Oefenweb/ansible-apparmor | Remove apparmor in Debian-like systems. |
Readings
- Ansible Directory Layout Best Practice
- Ansible doesn't load .profile and .bashrc files (21 Apr 2016)
- This is expected and by design, Ansible is a batch system and avoid interactive logins if possible, which means it won't source certain user/shell specific files.
- Using Ansible's command and shell modules properly (21 Sep 2016)
changed_when
,when
- File lookup() relative to playbook (Aug 27 '14)
- List of Ansible default variables
- Running Python script via ansible (Feb 1 '16)
- Ansible: copy template only when destination file does not exist (Jul 5 '14)
force: false
- Working with date and timestamp in Ansible (July 27, 2017)
- When command/shell outputs JSON, its result should be immediately parseable :
from_json
filter - Examples of using Blocks in Ansible 2 (March 8, 2016)
- "failed" attribute of a task result works strangely in conditionals (9 Dec 2015)
when: result|failed, when: result.failed, or when: result.failed == True
- ansible run command on remote host in background (Sep 6 '16)
- Using
async
andpoll
directives
- Using
- Filter/Test
- Ansible: filter a list by its attributes (Aug 8 '15)
network.addresses.private_man | selectattr("type", "match", "^fixed$") | map(attribute='addr') | list
- How to convert a dictionary of dictionaries into a list of dictionaries in a Ansible vars file? (May 17 '16)
hostvars.values()|list
- Looping through dictionaries in jinja2 templates (5 Nov 2015)
- Ansible/Jinja2 - Map nested key in list (Aug 14 '17)
sum(start=[])
- Ansible: filter a list by its attributes (Aug 8 '15)
- Map
- List and Dict Comprehensions in Python (21 February 2017)
- Mapping Types —
dict
- Security
- Ansible: How to encrypt some variables in an inventory file in a separate vault file? (May 13 '15)
ansible-vault
- Ansible: How to encrypt some variables in an inventory file in a separate vault file? (May 13 '15)
- Modules
- copy module contents parameter does not allow for newline chars (20 Feb 2014)
content: " {{ item | to_nice_json(indent=2) }}"
- ansible extract without first directory (Feb 20 '17)
extra_opts: [--strip-components=1]
- copy module contents parameter does not allow for newline chars (20 Feb 2014)
- Examples
Tips and Trics
Print out variables and expressions using Ansible console and debug module
Get into console using ansible-console
command and then use debug
task
$ ansible-console -l m001
...
sshuser@all (1)[f:5]$ debug msg="{{ hostvars }}"
...
sshuser@all (1)[f:5]$ exit
$
Safe escaping undefined variable
Use default
filter.
when: not fabric.generate.crypto.skip|default(false)
with_item: favorites.movies|default([])
{% for member in members|default([]) %}
{%- for message in (member.mailbox|default({})).messages|default([]) -%}
...
{%- endfor -%}
{% endfor %}
Filtering and drilling down using selectattr
and map
With 4 host variable files under host_vars
directory
host_vars/m001.yml
quorum:
nodes:
- name: node1
host: "{{ inventory_hostname }}"
type: permissioned
- name: node2
host: "{{ inventory_hostname }}"
type: permissioned
host_vars/m002yml
quorum:
nodes:
- name: node3
host: "{{ inventory_hostname }}"
type: permissioned
host_vars/m003yml
quorum:
portal:
host_vars/m004yml
grafana:
The following task would print out only 3 quorum nodes
- name: Print out Quorum nodes
debug:
msg: "{{ item }}"
with_items: "{{ hostvars.values()|selectattr('quorum', 'defined')|map(attribute='quorum')|selectattr('nodes', 'defined')|map(attribute='nodes')|sum(start=[])|list }}"
# when: (hostvars[item].quorum|default({})).nodes
when: true
Fabric
- http://www.fabfile.org/
- Desc. : a Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.
- License :
- Written in : Python
- Sources : https://github.com/fabric/fabric
- References
Companions
fabtools
- https://github.com/fabtools/fabtools
- Desc. : Tools for writing awesome Fabric files
- Readings
SaltStack
- https://saltstack.com/
- Desc. : Software to automate the management and configuration of any infrastructure or application at scale.
- Sources : https://github.com/saltstack/salt
- Readings
System Monitoring Software
- Prometheus vs. Graphite
- Prometheus vs. InfluxDB
- System Properties Comparison InfluxDB vs. Prometheus
- Prometheus vs InfluxDB
Ganglia
- http://ganglia.info/
- Desc : a scalable distributed monitoring system for high-performance computing systems such as clusters and grids.
- License : BSD
- Written in : C, Perl, PHP, Python
- Source : https://github.com/ganglia
- Readings
Graphite
- https://github.com/graphite-project
- Desc. : an enterprise-scale monitoring tool that runs well on cheap hardware.
- License : The Apache 2.0 License
- Written in : Python
- Sources : https://github.com/graphite-project
- Readings
Prometheus
- https://prometheus.io/
- Desc. : an open-source systems monitoring and alerting toolkit originally built at SoundCloud.
- License :
- Written in : Go
- Sources
- Distributions
- Docker Images
- References
- Configuration
<scrape_config>
: specifies a set of targets and parameters describing how to scrape them<file_sd_config>
: provides a more generic way to configure static targets and serves as an interface to plug in custom service discovery mechanisms.<relabel_config>
: a powerful tool to dynamically rewrite the label set of a target before it gets scraped.
- Querying
- Functions
- Exporters
- Configuration
- Readings
- Installing Prometheus Using Docker
- How To Use Prometheus to Monitor Your Ubuntu 14.04 Server (June 30, 2015)
- How To Set Up Prometheus To Monitor Your Ubuntu Server (Mar 15, 2016)
- How To Install Prometheus using Docker on Ubuntu 14.04(January 12, 2016)
- Monitoring linux stats with Prometheus.io
- SUSE System Monitoring Utilities
- Label
- Automatically generated labels and time series :
job
,instance
- Life of a Label(March 9, 2016)
- Controlling the instance label(August 15, 2016)
- Automatically generated labels and time series :
- Collectors and Metrics
Collector | Metrics | References | Remarks |
---|---|---|---|
meminfo
|
node_memory_MemTotal node_memory_MemFree node_memory_Buffers node_memory_Cached
|
CentOS > /proc/meminfo
|
/proc/meminfo
|
netdev
|
node_network_receive_bytes node_network_transmit_bytes
|
/proc/net/dev
|
- Metrics and Labels
Metric | Labels | Remarks |
---|---|---|
node_cpu
|
cpu , mode
|
|
node_disk_io+*
|
device
|
|
node_network_receive_bytes
|
device
|
- Typical Queries
Measure | Query | Remarks |
---|---|---|
Non-idle CPU usage of the specified physical node | 100 - avg(irate(node_cpu{job="node",instance="$instance",mode="idle"}[2m])) * 100
|
|
By-mode CPU usage of the specified physical node | (avg (irate(node_cpu{job="node",instance="$instance"}[2m])) by (mode)) * 100
|
|
Used memory(RAM) amount of the specified physical node | (node_memory_MemTotal{instance="$instance"} - (node_memory_MemFree{instance="$instance"} + node_memory_Buffers{instance="$instance"} + node_memory_Cached{instance="$instance"}))/ node_memory_MemTotal{instance="$instance"} * 100
|
|
Network sent traffic rate by interface of the specified physical node | rate(node_network_transmit_bytes{job="node",instance="$instance",device=~"lo|^eth.*"}[2m])
|
|
Network received traffic rate by interface of the specified physical node | rate(node_network_receive_bytes{job="node",instance="$instance",device=~"lo|^eth.*"}[2m])
|
Tips and Tricks
Available configuration flags of Node Exporter
"/usr/bin/prometheus-node-exporter -h
" yields the following.
-auth.pass string
Password for basic auth.
-auth.user string
Username for basic auth.
-collector.diskstats.ignored-devices string
Regexp of devices to ignore for diskstats. (default "^(ram|loop|fd|(h|s|v|xv)d[a-z])\\d+$")
-collector.filesystem.ignored-mount-points string
Regexp of mount points to ignore for filesystem collector. (default "^/(sys|proc|dev)($|/)")
-collector.ipvs.procfs string
procfs mountpoint. (default "/proc")
-collector.megacli.command string
Command to run megacli. (default "megacli")
-collector.netdev.ignored-devices string
Regexp of net devices to ignore for netdev collector. (default "^$")
-collector.ntp.server string
NTP server to use for ntp collector.
-collector.textfile.directory string
Directory to read text files with metrics from.
-collectors.enabled string
Comma-separated list of collectors to use. (default "diskstats,filesystem,loadavg,meminfo,stat,textfile,time,netdev,netstat")
-collectors.print
If true, print available collectors and exit.
-debug.memprofile-file string
Write memory profile to this file upon receipt of SIGUSR1.
-log.level value
Only log messages with the given severity or above. Valid levels: [debug, info, warn, error, fatal, panic]. (default info)
-web.listen-address string
Address on which to expose metrics and web interface. (default ":9100")
-web.telemetry-path string
Path under which to expose metrics. (default "/metrics")
Overriding instance
label
Candidate 1 : Using relabel_configs
- scrape_config:
job_name: node
static_configs:
- targets: ['10.0.0.1:9100/brick1' '10.0.02:9100/brick1']
relabel_configs:
- source_labels: [__address__]
target_label: instance
regex: ^[^/]*/(.*)
replacement: $1
- source_labels: [__address__]
target_label: __address__
regex: ^([^/]*).*
replacement: $1
Candidate 2 : Using lables
for each target
- scrape_config:
job_name: node
static_configs:
- targets: ['10.0.0.1:9100']
lables:
"instance":"brick1"
"role":"peer"
- targets: ['10.0.0.1:9100']
lables:
"instance":"brick1"
"role":"peer"
Companions
WMI Exporter
- https://github.com/martinlindhe/wmi_exporter
- Desc. : Prometheus exporter for Windows machines, using the WMI (Windows Management Instrumentation).
- License : The MIT License (MIT)
InfluxDB
- https://www.influxdata.com/
- Desc. : a time series database built from the ground up to handle high write and query loads
- License : MIT License
- Written in : Go
- Sources : https://github.com/influxdata/influxdb
- Docker Images
- Readings
Telegraf
- https://github.com/influxdata/telegraf
- Desc. : an agent written in Go for collecting, processing, aggregating, and writing metrics.
- License : MIT License
- Written in : Go
- Readings
Grafana
- https://grafana.com/
- Desc. : an open source metric analytics & visualization suite.
- License
- Sources : https://github.com/grafana/grafana
- Docker Image
- References
Wiki
DokuWiki
- https://www.dokuwiki.org/
- Desc. : a simple to use and highly versatile Open Source wiki software that doesn't require a database.
- License : GPL
- Written in : PHP
MoinMoin
- http://moinmo.in/
- Desc : an advanced, easy to use and extensible WikiEngine with a large community of users.
- License : GPL
gollum
- https://github.com/gollum/gollum
- Desc : a simple wiki system built on top of Git that powers GitHub Wikis.
- License : The MIT License
- Written in : Ruby
Content Management System
Jekyll
- http://jekyllrb.com/
- Desc. : a parsing engine bundled as a ruby gem used to build static websites from dynamic components such as templates, partials, liquid code, markdown, etc.
- License : The MIT License
- Written in : Ruby
- Sources : https://github.com/mojombo/jekyll
- References
- Readings
- Setting up your GitHub Pages site locally with Jekyll
bundle exec jekyll serve
- Jekyll style guide
- 5 Steps To Add Bootstrap 4 To Jekyll The Right Way
- Add Syntax Highlighting to your Jekyll site with Rouge (25 Mar 2017)
- Syntax Highlighting in Jekyll With Rouge (15 January 2015)
- Setting up your GitHub Pages site locally with Jekyll
Drupal
- http://drupal.org/
- Desc : a powerful content management system which allows you to create and maintain many different types of websites without needing to know any coding languages.
- License : GPL
Joomla
- http://www.joomla.org/
- Desc : a content management system (CMS), which enables you to build Web sites and powerful online applications.
- License : GPL
Tiki Wiki CMS Groupware
- http://info.tiki.org/
- Desc. : a free and open source wiki-based, content management system and Online office suite
- License : LGPL 2.1
- Written in : PHP
Desktop Sharing
x11vnc
- http://www.karlrunge.com/x11vnc/
- Desc. : a VNC server for real X displays
- License :
- Readings
TightVNC
- http://www.tightvnc.com/
- Desc. : a free remote control software package.
- License : GPL
BPM Engine
Activiti
- http://www.activiti.org/
- Desc : a light-weight workflow and Business Process Management (BPM) Platform targeted at business people, developers and system admins.
- License : Apache License 2.0
misc
Openfire
- http://www.igniterealtime.org/projects/openfire/
- Desc : a real time collaboration (RTC) server licensed under the Open Source Apache License.
- License : Apache License
Supervisor
- http://supervisord.org/
- Desc. : a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.
- License :
- Written in : Python
Spring Loaded
- https://github.com/spring-projects/spring-loaded
- Desc. : a JVM agent for reloading class file changes whilst a JVM is running.
- License : Apache License Version 2.0
Docker
- https://www.docker.com/
- Desc. : an open platform for developers and sysadmins to build, ship, and run distributed applications.
- License : Apache License Version 2.0
- Source : https://github.com/docker/docker
Vagrant
- https://www.vagrantup.com/
- Desc. : provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team.
- License :
Utilities
Browser
Firefox
- Readings
Thunderbird
- Readings
- Troubleshooting
PC Diagnostic and Recovery
GNU GRUB
- http://www.gnu.org/software/grub/
- Desc. : a Multiboot boot loader
- License : GPL 3+
Hiren's BootCD
- http://www.hiren.info/pages/bootcd
- Desc. : A bootable software CD containing a number of diagnostic programs such as partitioning agents, system performance benchmarks, disk cloning and imaging tools, data recovery tools, MBR tools, BIOS tools, and many others for fixing various computer problems.
- fan & discussion site (for download and more information)
- Articles
File Synchronization/Backup
rsync
- http://rsync.samba.org/
- Desc. : An open source utility that provides fast incremental file transfer.
- Front-end soft-wares or solutions based on rsync.
Partition Wizard
Optical Disc Authoring
CDBurnerXP
- http://cdburnerxp.se
- Desc. : Application to burn CDs and DVDs, including Blu-Ray and HD-DVDs.
- License
- Limited grants you (the licensee) a permission to use the software at no cost, both for commercial and non-commercial purposes on any computer in your possession.
InfraRecorder
- http://infrarecorder.org/
- Desc. : A free CD/DVD burning solution for Microsoft Windows.
- License : GPL
- InfraRecorder on Wikipedia
PC Security
Comodo Internet Security
- https://www.comodo.com/
- Desc. : an Internet security suite for Microsoft Windows including an antivirus program, a personal firewall, a sandbox and a host-based intrusion prevention system (HIPS)
- Comodo Internet Security on Wikipedia
Image Editor
GIMP
- https://www.gimp.org/
- Desc. : a cross-platform image editor available for GNU/Linux, OS X, Windows and more operating systems.
- License : GPL v3
- Readings
- Official user manual
- Move Tool : used to move layers, selections, paths or guides
- Floating selection in Gimp
- Align Visible Layers : can very precisely position the visible layers
- Removing Image Backgrounds - GIMP Fuzzy Select (31 Oct 2010)
misc
LibreOffice
- https://www.libreoffice.org/
- Desc. : a powerful office suite
- License : Mozilla Public License v2.0
Notepad++
- https://notepad-plus-plus.org/
- Desc. : a free source code editor and Notepad replacement that supports several languages.
- License : GPL
Excel
- Readings
CCleaner
- https://www.piriform.com/ccleaner
- Desc. : a utility program used to clean potentially unwanted files (including temporary internet files, where malicious programs and code tend to reside) and invalid Windows Registry entries from a computer.
- CCleaner on Wikipedia
Picasa
- Readings
Hangul Typing
misc
Standards
eTOM
- https://www.tmforum.org/business-process-framework/
- Desc. : a business process framework for telecom service providers in the telecommunications industry.
- Readings
ONVIF
- http://www.onvif.org/
- Desc. : an open industry forum for the development of a global standard for the interface of IP-based physical security products.
- References
- Specification
- ONVIF Specifications
- ONVIF Core Specification, ver 16.12
- Access Control, Access Rules, Action Engine, Advanced Security, Analytics, Credential, Device IO, Display, Door Control, Imaging, Media, Media2, Provisioning, PTZ, Receiver, Recording Control, Recording Search, Replay Control, Schedule, Thermal, Video Analytics Device
- ONVIF Core Specification, ver 2.5(Dec, 2014)
- Device Discovery, Device Management, Event Framework
- ONVIF Analytics Service Specification, ver 2.5(Dec, 2014)
- ONVIF Recording Search Service Specification, ver 2.5(Dec, 2014)
- Profile
- ONVIF Profile G Specification, ver 1.0(December 2013)
- Capabilities :
GetServices
,GetServiceCapabilities
- Recording Search :
FindRecordings
/GetRecordingSearchResults
,FindEvents
/GetEventSearchResults
- Recording Control :
CreateRecording
,DeleteRecording
,CreateTrack
,DeleteTrack
- Capabilities :
- ONVIF Profile G Specification, ver 1.0(December 2013)
- Schema
- WSLDs
- Specification
- Search
FindEvents
- starts a search session, looking for recording events in the scope that matches the search filter defined in the request.
GetEventSearchResults
- acquires the results from a recording event search session previously initiated by a FindEvents operation.
FindRecordings
- starts a search session, looking for recordings that matches the scope defined in the request.
GetRecordingSearchResults
- acquires the results from a recording search session previously initiated by a FindRecordings operation.
FindMetadata
- starts a search session, looking for metadata in the scope that matches the search filter defined in the request.
GetMetadataSearchResults
- acquires the results from a recording search session previously initiated by a FindMetadata operation.
Services
Amazon Redshift
- http://aws.amazon.com/redshift/
- Desc. : a fast, fully managed, petabyte-scale data warehouse solution that makes it simple and cost-effective to efficiently analyze all your data using your existing business intelligence tools.
- Readings
Microsoft Project Oxford
- https://www.projectoxford.ai/
- Desc. : a set of services for understanding data and adding ‘smart’ to your applications.
- Components
- Computer Vision APIs
- Face APIs
- Emotion APIs
- Speech APIs
- Spell Check APIs
- Language Understanding Intelligent Service
Awards Winner
JAX Innovation Awards
Categories | Winners | Remarks |
---|---|---|
Most innovative contribution to the Java ecosystem | Spring Boot | |
Most innovative solution to software delivery and DevOps | Docker | |
Special Jury Award | Let’s Encrypt |
Categories | Winners | Remarks |
---|---|---|
Most Innovative Java Technology | Java 8 | |
Most Innovative Open Tech | Akka | |
Special Jury Award | Netflix OSS |
Categories | Winners | Remarks |
---|---|---|
Most Innovative Java Technology | Vert.x | a tool-kit for building reactive applications on the JVM |
Most Innovative Open Technology | Docker | an open platform for distributed applications for developers and sysadmins |
Most Innovative Open Tech Business | Hazelcast | The Leading Open Source In-Memory Data Grid |
Categories | Winners | Remarks |
---|---|---|
Most Innovative Java Technology | Restructure101 | commercial product |
Most Innovative Java Company | JetBrains | |
Top Java Ambassador | Adam Bien | |
Special Jury Award | Charlie Nutter |
Categories | Winners | Remarks |
---|---|---|
Most Innovative Java Technology | JRebel | |
Most Innovative Java Company | Red Hat | |
Top Java Ambassador | Martin Odersky | |
Special Jury Award | Brian Goetz |
Software Ranking
Graphics Asset
- Tango Desktop Project
- http://tango.freedesktop.org/
- Defines an icon style guideline to which artists and designers can adhere and provides a sample implementation of the style as an icon theme based upon a standardized icon naming specification.
- Silk Icons
- http://www.famfamfam.com/lab/icons/silk/
- A smooth, free icon set, containing over 700 16-by-16 pixel icons in strokably-soft PNG format.
- Liquid Look And Feel
- http://sourceforge.net/projects/liquidlnf/
- Java2 Swing Look and Feel of Mosfet Liquid KDE 3.x theme.
- Open Source Web Design
- http://www.oswd.org/ http://www.oswd.org/
- A site to download free web design templates and share yours with others.
- IconFinder
- http://www.iconfinder.com/
- provides high quality icons for webdesigners and developers in an easy and efficient way.
- Openclipart
- https://openclipart.org/
- a community and collection of vector clip art that is free content
- Openclipart(Wikipedia)