Browse Source

A complete blog overhaul!

master
Elia el Lazkani 10 months ago
parent
commit
19ba03bdf0
  1. 5
      .drone.yml
  2. 1
      .gitignore
  3. 65
      conf.py
  4. 30
      files/assets/css/custom.css
  5. 23
      pages/about.org
  6. 20
      pages/about_me.rst
  7. 41
      pages/index.org
  8. 51
      pages/index.rst
  9. 15
      pages/not-found.org
  10. 12
      pages/not_found.rst
  11. 1
      plugins/__init__.py
  12. 52
      plugins/orgmode/README.md
  13. 8
      plugins/orgmode/conf.py.sample
  14. 1
      plugins/orgmode/emacs-htmlize
  15. 136
      plugins/orgmode/init.el
  16. 19
      plugins/orgmode/macros.org
  17. 15
      plugins/orgmode/orgmode.plugin
  18. 118
      plugins/orgmode/orgmode.py
  19. 2
      plugins/orgmode/requirements-nonpy.txt
  20. 237
      posts/backup/automating-borg.org
  21. 247
      posts/backup/automating-borg.rst
  22. 117
      posts/backup/borgbackup.org
  23. 131
      posts/backup/borgbackup.rst
  24. 480
      posts/configuration-management/ansible-testing-with-molecule.org
  25. 500
      posts/configuration-management/ansible_testing_with_molecule.rst
  26. 98
      posts/irc/weechat-ssh-and-notification.org
  27. 99
      posts/irc/weechat_ssh_and_notification.rst
  28. 163
      posts/k3s/building-k3s-on-a-pi.org
  29. 162
      posts/k3s/building_k3s_on_a_pi.rst
  30. 144
      posts/kubernetes/deploying-helm-in-your-kubernetes-cluster.org
  31. 145
      posts/kubernetes/deploying_helm_in_your_kubernetes_cluster.rst
  32. 229
      posts/kubernetes/local-kubernetes-cluster-on-kvm.org
  33. 224
      posts/kubernetes/local_kubernetes_cluster_on_kvm.rst
  34. 173
      posts/kubernetes/minikube-setup.org
  35. 179
      posts/kubernetes/minikube_setup.rst
  36. 388
      posts/kubernetes/your-first-minikube-helm-deployment.org
  37. 389
      posts/kubernetes/your_first_minikube_helm_deployment.rst
  38. 166
      posts/misc/a-quick-zfs-overview.org
  39. 173
      posts/misc/a_quick_zfs_overview.rst
  40. 112
      posts/misc/my-path-down-the-road-of-cloudflare-s-redirect-loop.org
  41. 104
      posts/misc/my_path_down_the_road_of_cloudflare_s_redirect_loop.rst
  42. 152
      posts/misc/the-story-behind-cmw.org
  43. 159
      posts/misc/the_story_behind_cmw.rst
  44. 126
      posts/monitoring/building-up-simple-monitoring-on-healthchecks.org
  45. 129
      posts/monitoring/building_up_simple_monitoring_on_healthchecks.rst
  46. 154
      posts/monitoring/simple-cron-monitoring-with-healthchecks.org
  47. 154
      posts/monitoring/simple_cron_monitoring_with_healthchecks.rst
  48. 230
      posts/revision-control/git-branching-and-merging.org
  49. 198
      posts/revision-control/git-first-steps.org
  50. 157
      posts/revision-control/git-rebase-and-strategies.org
  51. 189
      posts/revision-control/git-remote.org
  52. 238
      posts/revision-control/git_branching_and_merging.rst
  53. 218
      posts/revision-control/git_first_steps.rst
  54. 164
      posts/revision-control/git_rebase_and_strategies.rst
  55. 193
      posts/revision-control/git_remotes.rst
  56. 236
      posts/text-editors/emacs-and-org-mode.org
  57. 241
      posts/text-editors/emacs-and-org-mode.rst
  58. 10751
      themes/custom/assets/css/bootstrap.css
  59. 12
      themes/custom/assets/css/bootstrap.min.css
  60. 12
      themes/custom/custom.theme
  61. 1
      themes/willy-theme

5
.drone.yml

@ -5,7 +5,7 @@ name: Deploy
steps:
- name: build
image: nikola:20200217
image: nikola:20200831
volumes:
- name: blog-path
path: /drone/src/output/
@ -21,5 +21,4 @@ steps:
volumes:
- name: blog-path
host:
path: /mnt/blog/
path: /mnt/new_blog/

1
.gitignore

@ -3,6 +3,7 @@ __pycache__/
output/
cache/
.doit.db
.doit.db.*
.venv/
.DS_Store
.venv

65
conf.py

@ -25,7 +25,8 @@ SOURCE_CODE_URL = "https://gitea.project42.io/Elia/blog.lazkani.io"
# This is the URL where Nikola's output will be deployed.
# If not set, defaults to SITE_URL
# BASE_URL = "https://blog.lazkani.io/"
BLOG_EMAIL = None
#BLOG_EMAIL = None
BLOG_EMAIL = ""
BLOG_DESCRIPTION = "A blog where technology, passion and experience combine" # (translatable)
# Nikola is multilingual!
@ -139,16 +140,12 @@ TRANSLATIONS_PATTERN = '{path}.{lang}.{ext}'
NAVIGATION_LINKS = {
DEFAULT_LANG: (
(
(
("/blog/", "Blog"),
("/archive.html", "Archives"),
("/categories/", "Tags"),
("/rss.xml", "RSS feed")
),
"Blog Content"
),
("/about_me/", "About me"),
("/", "Home"),
("/blog/", "Blog Posts"),
("/archive.html", "Archives"),
("/categories/", "Tags"),
("/rss.xml", "RSS feed"),
("/about/", "About"),
),
}
@ -161,7 +158,8 @@ NAVIGATION_ALT_LINKS = {
# Name of the theme to use.
#THEME = "bootblog4"
THEME = "custom"
#THEME = "custom"
THEME = "willy-theme"
# Primary color of your theme. This will be used to customize your theme.
# Must be a HEX value.
@ -229,12 +227,14 @@ POSTS = (
("posts/*.md", "posts", "post.tmpl"),
("posts/*.txt", "posts", "post.tmpl"),
("posts/*.html", "posts", "post.tmpl"),
("posts/*.org", "posts", "post.tmpl"),
)
PAGES = (
("pages/*.rst", "", "page.tmpl"),
("pages/*.md", "", "page.tmpl"),
("pages/*.txt", "", "page.tmpl"),
("pages/*.html", "", "page.tmpl"),
("pages/*.org", "", "page.tmpl"),
)
@ -313,6 +313,7 @@ COMPILERS = {
# but is disabled by default as it would conflict
# with many of the others.
# "pandoc": ('.rst', '.md', '.txt'),
"orgmode": ('.org',),
}
# Create by default posts in one file format?
@ -652,7 +653,7 @@ GITHUB_REMOTE_NAME = 'origin'
# Whether or not github_deploy should commit to the source branch automatically
# before deploying.
GITHUB_COMMIT_SOURCE = True
GITHUB_COMMIT_SOURCE = False
# Where the output site should be located
# If you don't use an absolute path, it will be considered as relative
@ -941,6 +942,7 @@ LICENSE = """
# A small copyright notice for the page footer (in HTML).
# (translatable)
CONTENT_FOOTER = '''
<center>
Stay in touch
@ -952,8 +954,11 @@ Stay in touch
<a href="https://github.com/elazkani" title="My GitHub Profile" class="fab fa-github" target="_blank"></a>
<br />
<br />
Contents &copy; {date} <a href="{source_code_url}">{author}</a> - Powered by <a href="https://getnikola.com" rel="nofollow">Nikola</a> under {license}
Contents &copy; {date} <a href="{source_code_url}">{author}</a>
<br />
Powered by <a href="https://getnikola.com" rel="nofollow">Nikola</a>
<br />
{license}
</center>
'''
@ -1163,7 +1168,7 @@ GENERATE_ATOM = True
# FEED_TEASERS = True
# Strip HTML from Atom and RSS feed summaries and content. Defaults to False.
# FEED_PLAIN = False
FEED_PLAIN = True
# Number of posts in Atom and RSS feeds.
# FEED_LENGTH = 10
@ -1183,20 +1188,20 @@ GENERATE_ATOM = True
# This search form works for any site and looks good in the "site" theme where
# it appears on the navigation bar:
SEARCH_FORM = """
<!-- DuckDuckGo custom search -->
<form method="get" id="search" action="https://duckduckgo.com/"
class="navbar-form pull-left">
<input type="hidden" name="sites" value="%s">
<input type="hidden" name="k8" value="#444444">
<input type="hidden" name="k9" value="#D51920">
<input type="hidden" name="kt" value="h">
<input type="text" name="q" maxlength="255"
placeholder="Search&hellip;" class="span2" style="margin-top: 4px;">
<input type="submit" value="DuckDuckGo Search" style="visibility: hidden;">
</form>
<!-- End of custom search -->
""" % SITE_URL
#SEARCH_FORM = """
#<!-- DuckDuckGo custom search -->
#<form method="get" id="search" action="https://duckduckgo.com/"
# class="navbar-form pull-left">
#<input type="hidden" name="sites" value="%s">
#<input type="hidden" name="k8" value="#444444">
#<input type="hidden" name="k9" value="#D51920">
#<input type="hidden" name="kt" value="h">
#<input type="text" name="q" maxlength="255"
# placeholder="Search&hellip;" class="span2" style="margin-top: 4px;">
#<input type="submit" value="DuckDuckGo Search" style="visibility: hidden;">
#</form>
#<!-- End of custom search -->
#""" % SITE_URL
# If you prefer a Google search form, here's an example that should just work:
# SEARCH_FORM = """
@ -1231,10 +1236,12 @@ SEARCH_FORM = """
# before </head>
# (translatable)
# EXTRA_HEAD_DATA = ""
EXTRA_HEAD_DATA = """<link rel="stylesheet" href="/assets/css/index.css">"""
# Google Analytics or whatever else you use. Added to the bottom of <body>
# in the default template (base.tmpl).
# (translatable)
# BODY_END = ""
BODY_END = """<script src="/assets/js/index.js"></script>"""
# The possibility to extract metadata from the filename by using a
# regular expression.

30
files/assets/css/custom.css

@ -1,30 +0,0 @@
.literal {
border: 1px solid #ccc;
color: #999;
background-color: #272822;
border-radius: 3px;
font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
white-space: nowrap;
font-size: 12px;
padding: 2px 4px;
}
div.note {
word-wrap: break-word;
background-color: rgb(34,34,34);
border: 1px solid #007053;
}
div.admonition, div.hint, div.important, div.note, div.tip, div.sidebar, div.attention, div.caution, div.danger, div.error, div.warning, div.system-message {
background-color: rgb(34,34,34);
}
div.note p.admonition-title {
color: #ffffff;
background-color: #007053 !important;
border-bottom: 1px solid #007053;
}
div.admonition p, div.hint p, div.important p, div.note p, div.tip p, div.sidebar p, div.attention p, div.caution p, div.danger p, div.error p, div.warning p, div.system-message p {
color: #ffffff;
}

23
pages/about.org

@ -0,0 +1,23 @@
#+BEGIN_COMMENT
.. title: About
.. date: 2019-06-21T00:00:00+02:00
.. author: Elia el Lazkani
.. url: about
.. status: published
#+END_COMMENT
* Who am I ?
I am a DevOps engineer with a passion for technology, automation, Linux and OpenSource. I love learning new tricks and challenging myself with new tools being released on a monthly bases around /kubernetes/ and/or /configuration management/. On my free time, I like to write automation tools and packages which can be found on PyPI. Or, I might as well tinker with new things around /kubernetes/. I blog about all that /here/. I think if I can write a blog about it, I understand it enough to have an opinion about it. It all comes in handy when the business need arises. I play around with technologies all day long by deploying, configuring, managing and maintaining all parts of the infrastructure below the application layer. I dabbled with "architecting" parts of different infrastructures, from end to end and I can say I have a knack for it and I like it when possible.
* Experience
Here's a quick and dirty list of some of the technologies I've had my hands dirty with.
- *Neworking*: Configuring routers and switches (Brocade, CISCO, Dell).
- *Infrastructure*: Finding, automating, deploying and managing infrastructure key services. Too many to mention.
- *Virtualization*: Building infrastructures for virtualization (HyperV, libvirt, proxmox, RHEV, VMWare).
- *Configuration Management*: Ansible, Chef, Puppet, SaltStack.
- *CI/CD*: Gitlab-CI, Jenkins.
- *Cloud*: AWS, Linode & Hetzner.
- *Development*: Python and Golang packages for plugging in different technologies together for automation.
- *Containers*: Docker, K8s and k3s deployment, management and supporting team deployments.
- *Misc*: Service meshes, Hashistack and misc other tools

20
pages/about_me.rst

@ -1,20 +0,0 @@
.. title: About me
.. date: 2019-06-21
.. status: published
.. authors: Elia el Lazkani
I am a DevOps engineer with a passion for technology, automation, Linux and OpenSource. I love learning new tricks and challenging myself with new tools being released on a monthly bases around *kubernetes* and/or *configuration management*. On my free time, I like to write automation tools and packages which can be found on PyPI. Or, I might as well tinker with new things around *kubernetes*. I blog about all that *here*. I think if I can write a blog about it, I understand it enough to have an opinion about it. It all comes in handy when the business need arises. I play around with technologies all day long by deploying, configuring, managing and maintaining all parts of the infrastructure below the application layer. I dabbled with "architecting" parts of different infrastructures, from end to end and I can say I have a knack for it and I like it when possible.
Experience
==========
Here's a quick and dirty list of some of the technologies I've had my hands dirty with.
- **Neworking**: Configuring routers and switches (Brocade, CISCO, Dell).
- **Infrastructure**: Finding, automating, deploying and managing infrastructure key services. Too many to mention.
- **Virtualization**: Building infrastructures for virtualization (HyperV, libvirt, proxmox, RHEV, VMWare).
- **Configuration Management**: Ansible, Chef, Puppet.
- **CI/CD**: Gitlab-CI, Jenkins.
- **Cloud**: AWS.
- **Development**: Python packages for plugging in different technologies together for automation.
- **Containers**: Docker and Kubernetes deployment, management and supporting team deployments.

41
pages/index.org

@ -0,0 +1,41 @@
#+BEGIN_COMMENT
.. title: Welcome to the DevOps Blog
.. date: 2019-06-23T00:00:00+02:00
.. status: published
.. slug: index
.. type: text
#+END_COMMENT
* What is this ?
This is my humble blog where I post things related to DevOps in hope that I or someone else might benefit from it.
* Wait what ? What is DevOps ?
[[https://duckduckgo.com/?q=what+is+devops+%3F&t=ffab&ia=web&iax=about][Duckduckgo]] defines DevOps as:
#+BEGIN_QUOTE
DevOps is a software engineering culture and practice that aims at unifying
software development and software operation. The main characteristic of the
DevOps movement is to strongly advocate automation and monitoring at all
steps of software construction, from integration, testing, releasing to
deployment and infrastructure management. DevOps aims at shorter development
cycles, increased deployment frequency, and more dependable releases,
in close alignment with business objectives.
#+END_QUOTE
In short, we build an infrastructure that is easily deployable, maintainable and, in all forms, makes the lives of the developers a breeze.
* What do you blog about ?
Anything and everything related to DevOps. The field is very big and complex with a lot of different tools and technologies implemented.
I try to blog about interesting and new things as much as possible, when time permits.
Here's a short list of the latest posts.
{{% post-list start=0 stop=3 %}}
{{% /post-list %}}
* Projects
- [[https://gitea.project42.io/Elia/blog.lazkani.io][blog.lazkani.io]]: The DevOps [[https://blog.lazkani.io][blog]]
- [[https://gitlab.com/elazkani/weenotify][weenotify]]: an official [[https://weechat.org][weechat]] notification plugin.
- [[https://gitlab.com/elazkani/rundeck-resources][rundeck-resources]]: python tool to query resources from different sources and export them into a data structure that [[https://www.rundeck.com/open-source][Rundeck]] can consume. This tool can be found on [[https://pypi.org/project/rundeck-resources/][PyPI]].
- [[https://gitlab.com/elazkani/get-k8s-resources][get\under{}k8s\under{}resources]]: a small python script that returns a list of kubernetes resources.

51
pages/index.rst

@ -1,51 +0,0 @@
.. title: Welcome to the DevOps blog
.. slug: index
.. date: 2019-06-23
.. tags:
.. category:
.. description:
.. type: text
What is this ?
==============
This is my humble blog where I post things related to DevOps in hope that I or someone else might benefit from it.
Wait what ? What is DevOps ?
============================
`Duckduckgo <https://duckduckgo.com/?q=what+is+devops+%3F&t=ffab&ia=web&iax=about>`_ define DevOps as:
DevOps is a software engineering culture and practice that aims at unifying
software development and software operation. The main characteristic of the
DevOps movement is to strongly advocate automation and monitoring at all
steps of software construction, from integration, testing, releasing to
deployment and infrastructure management. DevOps aims at shorter development
cycles, increased deployment frequency, and more dependable releases,
in close alignment with business objectives.
In short, we build an infrastructure that is easily deployable, maintainable and, in all forms, makes the lives of the developers a breeze.
What do you blog about ?
========================
Anything and everything related to DevOps. The field is very big and complex with a lot of different tools and technologies implemented.
I try to blog about interesting and new things as much as possible, when time permits.
Here's a short list of the latest posts.
.. post-list::
:start: 0
:stop: 3
Projects
========
- `blog.lazkani.io <https://gitea.project42.io/Elia/blog.lazkani.io>`_: The DevOps `blog <https://blog.lazkani.io>`_.
- `weenotify <https://gitlab.com/elazkani/weenotify>`_: an official `weechat <https://weechat.org>`_ notification plugin.
- `rundeck-resources <https://gitlab.com/elazkani/rundeck-resources>`_: python tool to query resources from different sources and export them into a data structure that `Rundeck <https://www.rundeck.com/open-source>`_ can consume. This tool can be found on `PyPI <https://pypi.org/project/rundeck-resources/>`_.
- `get_k8s_resources <https://gitlab.com/elazkani/get-k8s-resources>`_: a small python script that returns a list of kubernetes resources.

15
pages/not-found.org

@ -0,0 +1,15 @@
#+BEGIN_COMMENT
.. title: Not Found
.. slug: not-found
.. date: 2020-02-08
.. tags:
.. category:
.. description:
.. type: text
#+END_COMMENT
Oops... We don't know how you ended up here.
There is nothing here to look at...
The main site is over @ {{% doc name="test" %}}index{{% /doc %}}.

12
pages/not_found.rst

@ -1,12 +0,0 @@
.. title: Not Found
.. slug: not-found
.. date: 2020-02-08
.. tags:
.. category:
.. description:
.. type: text
Oops... We don't know how you ended up here.
There is nothing here to look at, try the main :doc:`site <index>`.

1
plugins/__init__.py

@ -0,0 +1 @@
# Plugin modules go here.

52
plugins/orgmode/README.md

@ -0,0 +1,52 @@
This plugin implements an Emacs Org-mode based compiler for Nikola.
## Setup
If your emacs does not ship with org-mode (>=8.x), you will have to edit the
`init.el` file supplied with this plugin, and load a newer version of org-mode.
You will also need to add the orgmode compiler to your list of compilers, and
modify your POSTS & PAGES variables. (See the sample conf file provided.)
### Syntax highlighting with pygments
By default, the plugin uses `pygments` for syntax highlighting. You can disable
this by setting `nikola-use-pygments` to `nil` in `init.el` or `conf.el` (see
Customization section below).
To get proper syntax highlighting, you will need to add custom CSS to your
theme. You can generate this CSS using the `pygmentize` command as follows:
mkdir -p files/assets/css
pygmentize -S <PYGMENTS_STYLE> -a .highlight -f html >> files/assets/css/custom.css
and make sure that `custom.css` is included in your site by your
theme. The various available style options for `<PYGMENTS_STYLE>` can be found
using the command `pygmentize -L style`.
## Customization
You can add any customization variables that you wish to add, to modify the
output generated by org-mode to `conf.el` inside the plugin directory. This
lets you have custom configuration, that doesn't get wiped out each time the
plugin is updated.
## Teasers
You may use teasers by enabling `INDEX_TEASERS = True` in conf.py, and
use `{{{TEASER_END}}}` to generate `<!-- TEASER_END -->` in org posts.
## Image URLs
The image url in ox-html is a little fuzzy. For example, `[[/images/test.jpg]]` will be
generated as `<img src="file:///images/test.jpg" alt="test.jpg">`
because the path is considered as an absolute file path.
If you add inline images with built-in file scheme to see them on your
Emacs orgmode, the references are resolved. For examples, you may
refer with `[[file:../images/test.jpg]]` to review an image stored on
images directory during writing, then it will be generated as `<img
src="images/test.jpg" alt="test.jpg">`.
Alternatively, you may write `[[img-url:/images/test.jpg]]`, and then
it should be generated as `<img src="/images/test.jpg" alt="test.jpg">`.

8
plugins/orgmode/conf.py.sample

@ -0,0 +1,8 @@
# NOTE: Needs additional configuration in init.el file.
# Add the orgmode compiler to your COMPILERS dict.
COMPILERS["orgmode"] = ['.org']
# Add org files to your POSTS, PAGES
POSTS = POSTS + (("posts/*.org", "posts", "post.tmpl"),)
PAGES = PAGES + (("pages/*.org", "pages", "page.tmpl"),)

1
plugins/orgmode/emacs-htmlize

@ -0,0 +1 @@
Subproject commit 49205105898ba8993b5253beec55d8bddd820a70

136
plugins/orgmode/init.el

@ -0,0 +1,136 @@
;; Init file to use with the orgmode plugin.
;; Load org-mode
;; Requires org-mode v8.x
(require 'package)
(add-to-list 'load-path "~/.emacs.d/.local/straight/build/htmlize")
(setq package-load-list '((htmlize t)))
(package-initialize)
(require 'org)
(require 'ox-html)
;;; Custom configuration for the export.
;;; Add any custom configuration that you would like to 'conf.el'.
(setq nikola-use-pygments t
org-export-with-toc nil
org-export-with-section-numbers nil
org-startup-folded 'showeverything)
;; Load additional configuration from conf.el
(let ((conf (expand-file-name "conf.el" (file-name-directory load-file-name))))
(if (file-exists-p conf)
(load-file conf)))
;;; Macros
;; Load Nikola macros
(setq nikola-macro-templates
(with-current-buffer
(find-file
(expand-file-name "macros.org" (file-name-directory load-file-name)))
(org-macro--collect-macros)))
;;; Code highlighting
(defun org-html-decode-plain-text (text)
"Convert HTML character to plain TEXT. i.e. do the inversion of
`org-html-encode-plain-text`. Possible conversions are set in
`org-html-protect-char-alist'."
(mapc
(lambda (pair)
(setq text (replace-regexp-in-string (cdr pair) (car pair) text t t)))
(reverse org-html-protect-char-alist))
text)
;; Use pygments highlighting for code
(defun pygmentize (lang code)
"Use Pygments to highlight the given code and return the output"
(with-temp-buffer
(insert code)
(let ((lang (or (cdr (assoc lang org-pygments-language-alist)) "text")))
(shell-command-on-region (point-min) (point-max)
(format "pygmentize -f html -l %s" lang)
(buffer-name) t))
(buffer-string)))
(defconst org-pygments-language-alist
'(("asymptote" . "asymptote")
("awk" . "awk")
("c" . "c")
("c++" . "cpp")
("cpp" . "cpp")
("clojure" . "clojure")
("css" . "css")
("d" . "d")
("emacs-lisp" . "scheme")
("F90" . "fortran")
("gnuplot" . "gnuplot")
("groovy" . "groovy")
("haskell" . "haskell")
("java" . "java")
("js" . "js")
("julia" . "julia")
("latex" . "latex")
("lisp" . "lisp")
("makefile" . "makefile")
("matlab" . "matlab")
("mscgen" . "mscgen")
("ocaml" . "ocaml")
("octave" . "octave")
("perl" . "perl")
("picolisp" . "scheme")
("python" . "python")
("r" . "r")
("ruby" . "ruby")
("sass" . "sass")
("scala" . "scala")
("scheme" . "scheme")
("sh" . "sh")
("sql" . "sql")
("sqlite" . "sqlite3")
("tcl" . "tcl"))
"Alist between org-babel languages and Pygments lexers.
lang is downcased before assoc, so use lowercase to describe language available.
See: http://orgmode.org/worg/org-contrib/babel/languages.html and
http://pygments.org/docs/lexers/ for adding new languages to the mapping.")
;; Override the html export function to use pygments
(defun org-html-src-block (src-block contents info)
"Transcode a SRC-BLOCK element from Org to HTML.
CONTENTS holds the contents of the item. INFO is a plist holding
contextual information."
(if (org-export-read-attribute :attr_html src-block :textarea)
(org-html--textarea-block src-block)
(let ((lang (org-element-property :language src-block))
(code (org-element-property :value src-block))
(code-html (org-html-format-code src-block info)))
(if nikola-use-pygments
(progn
(unless lang (setq lang ""))
(pygmentize (downcase lang) (org-html-decode-plain-text code)))
code-html))))
;; Export images with custom link type
(defun org-custom-link-img-url-export (path desc format)
(cond
((eq format 'html)
(format "<img src=\"%s\" alt=\"%s\"/>" path desc))))
(org-add-link-type "img-url" nil 'org-custom-link-img-url-export)
;; Export images with built-in file scheme
(defun org-file-link-img-url-export (path desc format)
(cond
((eq format 'html)
(format "<img src=\"/%s\" alt=\"%s\"/>" path desc))))
(org-add-link-type "file" nil 'org-file-link-img-url-export)
;; Export function used by Nikola.
(defun nikola-html-export (infile outfile)
"Export the body only of the input file and write it to
specified location."
(with-current-buffer (find-file infile)
(org-macro-replace-all nikola-macro-templates)
(org-html-export-as-html nil nil t t)
(write-file outfile nil)))

19
plugins/orgmode/macros.org

@ -0,0 +1,19 @@
# Macros for embedding media into org-mode posts.
#+MACRO: TEASER_END #+HTML: <!-- TEASER_END -->
{{{TEASER_END}}}
#+MACRO: gist #+HTML: <script src="https://gist.github.com/$1.js"></script>
{{{gist(2395294)}}}
#+MACRO: soundcloud #+HTML: <iframe width="$3" height="$2" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https://api.soundcloud.com/tracks/$1"> </iframe>
{{{soundcloud(31824842,240,320)}}}
#+MACRO: soundcloud_playlist #+HTML: <iframe width="$3" height="$2" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https://api.soundcloud.com/playlists/$1"> </iframe>
{{{soundcloud_playlist(694081,800,400)}}}
#+MACRO: vimeo #+HTML: <iframe src="https://player.vimeo.com/video/$1" width="$3" height="$2" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen> </iframe>
{{{vimeo(85360039,240,320)}}}
#+MACRO: youtube #+HTML: <iframe width="$3" height="$2" src="https://www.youtube.com/embed/$1?rel=0&amp;hd=1&amp;wmode=transparent"></iframe>
{{{youtube(8N_tupPBtWQ,240,320)}}}

15
plugins/orgmode/orgmode.plugin

@ -0,0 +1,15 @@
[Core]
Name = orgmode
Module = orgmode
[Nikola]
MinVersion = 6.0.0
PluginCategory = PageCompiler
[Documentation]
Author = Puneeth Chaganti
Version = 0.3
Website = http://plugins.getnikola.com/#orgmode
Description = Compile org-mode markup into HTML using emacs.

118
plugins/orgmode/orgmode.py

@ -0,0 +1,118 @@
# -*- coding: utf-8 -*-
# Copyright © 2012-2013 Puneeth Chaganti and others.
# Permission is hereby granted, free of charge, to any
# person obtaining a copy of this software and associated
# documentation files (the "Software"), to deal in the
# Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the
# Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice
# shall be included in all copies or substantial portions of
# the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
# OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
""" Implementation of compile_html based on Emacs Org-mode.
You will need to install emacs and org-mode (v8.x or greater).
"""
from __future__ import unicode_literals
import io
import os
from os.path import abspath, dirname, join
import shlex
import subprocess
try:
from collections import OrderedDict
except ImportError:
OrderedDict = dict # NOQA
from nikola.plugin_categories import PageCompiler
from nikola.utils import req_missing, makedirs
# v6 compat
try:
from nikola.utils import write_metadata
except ImportError:
write_metadata = None # NOQA
class CompileOrgmode(PageCompiler):
""" Compile org-mode markup into HTML using emacs. """
name = "orgmode"
def compile(self, source, dest, is_two_file=True, post=None, lang=None):
"""Compile the source file into HTML and save as dest."""
makedirs(os.path.dirname(dest))
try:
command = [
'emacs', '--batch',
'-l', join(dirname(abspath(__file__)), 'init.el'),
'--eval', '(nikola-html-export "{0}" "{1}")'.format(
abspath(source), abspath(dest))
]
# Dirty walkaround for this plugin to run on Windows platform.
if os.name == 'nt':
command[5] = command[5].replace("\\", "\\\\")
subprocess.check_call(command)
with io.open(dest, 'r', encoding='utf-8') as inf:
output, shortcode_deps = self.site.apply_shortcodes(
inf.read(), extra_context={'post': post})
with io.open(dest, 'w', encoding='utf-8') as outf:
outf.write(output)
if post is None:
if shortcode_deps:
self.logger.error(
"Cannot save dependencies for post {0} (post unknown)",
source)
else:
post._depfile[dest] += shortcode_deps
except OSError as e:
import errno
if e.errno == errno.ENOENT:
req_missing(['emacs', 'org-mode'],
'use the orgmode compiler', python=False)
except subprocess.CalledProcessError as e:
raise Exception('''Cannot compile {0} -- bad org-mode configuration (return code {1})
The command is {2}'''.format(source, e.returncode, ' '.join(shlex.quote(arg) for arg in e.cmd)))
def create_post(self, path, content=None, onefile=False, is_page=False, **kw):
"""Create post file with optional metadata."""
metadata = OrderedDict()
metadata.update(self.default_metadata)
metadata.update(kw)
makedirs(os.path.dirname(path))
with io.open(path, "w+", encoding="utf-8") as fd:
if onefile:
fd.write("#+BEGIN_COMMENT\n")
if write_metadata:
fd.write(write_metadata(metadata))
else:
for k, v in metadata.items():
fd.write('.. {0}: {1}\n'.format(k, v))
fd.write("#+END_COMMENT\n")
fd.write("\n\n")
if content:
fd.write(content)
else:
fd.write('Write your post here.')

2
plugins/orgmode/requirements-nonpy.txt

@ -0,0 +1,2 @@
Emacs::https://www.gnu.org/software/emacs/
Org-mode::http://orgmode.org/

237
posts/backup/automating-borg.org

@ -0,0 +1,237 @@
#+BEGIN_COMMENT
.. title: Automating Borg
.. date: 2020-02-02
.. slug: automating-borg
.. updated: 2020-02-02
.. status: published
.. tags: backup, borgbackup, borg, borgmatic
.. category: backup
.. authors: Elia el Lazkani
.. description: We've had a look at **Borg** before, let's find out how to automate it.
.. type: text
#+END_COMMENT
In the previous blog post entitle "{{% doc %}}borgbackup{{% /doc %}}, I talked about *borg*.
If you read that post, you would've noticed that *borg* has a lot of features.
With a lot of features come a lot of automation.
If you were thinking about using *borg*, you should either make a /simple cron/ or you're gonna have to write an elaborate script to take care of all the different steps.
What if I told you there's another way ? An easier way ! The *Borgmatic* way... What would you say ?
* Borgmatic
*Borgmatic* is defined on their [[https://torsion.org/borgmatic/][website]] as follows.
#+BEGIN_QUOTE
borgmatic is simple, configuration-driven backup software for servers
and workstations. Protect your files with client-side encryption.
Backup your databases too. Monitor it all with integrated third-party
services.
#+END_QUOTE
If you go down to it, *borgmatic* uses *borg*'s /API/ to automate a list of configurable /tasks/.
This way, it saves you the trouble of writing your own scripts to automate these steps.
*Borgmatic* uses a /YAML/ configuration file. Let's configure a few tasks.
* Location
First, let's start by configuring the locations that *borg* is going to be working with.
#+BEGIN_SRC yaml
location:
source_directories:
- /home/
repositories:
- user@backupserver:sourcehostname.borg
one_file_system: true
exclude_patterns:
- /home/*/.cache
- '*.pyc'
#+END_SRC
This tells *borg* that we need to backup our =/home= directories excluding a few patterns.
Let's not forget that we told *borg* where the repository is located at.
* Storage
We need to configure the storage next.
#+BEGIN_SRC yaml
storage:
# Recommended
# encryption_passcommand: secret-tool lookup borg-repository repo-name
encryption_passphrase: "ReallyStrongPassphrase"
compression: zstd,15
ssh_command: ssh -i /path/to/private/key
borg_security_directory: /path/to/base/config/security
archive_name_format: 'borgmatic-{hostname}-{now}'
#+END_SRC
In this section, we tell borg a little big of information about our repository.
What are the credentials, where it can find them, etc.
The easy way is to go with a =passphrase=, but I recommend using an =encryption_passcommand= instead.
I also use =zstd= for encryption instead of =lz4=, you better do your research before you change the default.
I also recommend, just as they do, the use of a security directory as well.
* Retention
We can configure a retention for our backups, if we like.
#+BEGIN_SRC yaml
retention:
keep_hourly: 7
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 2
prefix: "borgmatic-"
#+END_SRC
The part of what to keep from /hourly/ to /daily/ is self explanatory.
I would like to point out the =prefix= part as it is important.
This is the /prefix/ that *borgmatic* uses to consider backups for *pruning*.
#+BEGIN_EXPORT html
<div class="admonition warning">
<p class="admonition-title">warning</p>
#+END_EXPORT
Watch out for the retention =prefix=
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
* Consistency
After the updates, we'd like to check our backups.
#+BEGIN_SRC yaml
consistency:
checks:
- repository
- archives
check_last: 3
prefix: "borgmatic-"
#+END_SRC
#+BEGIN_EXPORT html
<div class="admonition warning">
<p class="admonition-title">warning</p>
#+END_EXPORT
Watch out, again, for the consistency =prefix=
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
* Hooks
Finally, hooks.
I'm going to talk about hooks a bit. Hooks can be used to backup *MySQL*, *PostgreSQL* or *MariaDB*.
They can also be hooks for =on_error=, =before_backup=, =after_backup=, =before_everything= and =after_everything=.
You can also hook to third party services which you can check on their webpage.
I deployed my own, so I configured my own.
* Borgmatic Configuration
Let's put everything together now.
#+BEGIN_SRC yaml
location:
source_directories:
- /home/
repositories:
- user@backupserver:sourcehostname.borg
one_file_system: true
exclude_patterns:
- /home/*/.cache
- '*.pyc'
storage:
# Recommended
# encryption_passcommand: secret-tool lookup borg-repository repo-name
encryption_passphrase: "ReallyStrongPassphrase"
compression: zstd,15
ssh_command: ssh -i /path/to/private/key
borg_security_directory: /path/to/base/config/security
archive_name_format: 'borgmatic-{hostname}-{now}'
retention:
keep_hourly: 7
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 2
prefix: "borgmatic-"
consistency:
checks:
- repository
- archives
check_last: 3
prefix: "borgmatic-"
#+END_SRC
Now that we have everything together, let's save it in =/etc/borgmatic.d/home.yaml=.
* Usage
If you have *borg* and *borgmatic* already installed on your system and the *borgmatic* configuration file in place, you can test it out.
You can create the repository.
#+BEGIN_EXAMPLE
# borgmatic init -v 2
#+END_EXAMPLE
You can list the backups for the repository.
#+BEGIN_EXAMPLE
# borgmatic list --last 5
borgmatic-home-2020-01-30T22:01:30 Thu, 2020-01-30 22:01:42 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-01-31T22:02:12 Fri, 2020-01-31 22:02:24 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-01T22:01:34 Sat, 2020-02-01 22:01:45 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-02T16:01:22 Sun, 2020-02-02 16:01:32 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-02T18:01:36 Sun, 2020-02-02 18:01:47 [0000000000000000000000000000000000000000000000000000000000000000]
#+END_EXAMPLE
You could run a check.
#+BEGIN_EXAMPLE
# borgmatic check -v 1
/etc/borgmatic.d/home.yaml: Pinging Healthchecks start
/borg/home: Running consistency checks
Remote: Starting repository check
Remote: Starting repository index check
Remote: Completed repository check, no problems found.
Starting archive consistency check...
Analyzing archive borgmatic-home-2020-02-01T22:01:34 (1/3)
Analyzing archive borgmatic-home-2020-02-02T16:01:22 (2/3)
Analyzing archive borgmatic-home-2020-02-02T18:01:36 (3/3)
Orphaned objects check skipped (needs all archives checked).
Archive consistency check complete, no problems found.
summary:
/etc/borgmatic.d/home.yaml: Successfully ran configuration file
#+END_EXAMPLE
But most of all, if you simply run =borgmatic= without any parameters, it will run through the whole configuration and apply all the steps.
At this point, you can simply add the =borgmatic= command in a *cron* to run on an interval.
The other options would be to configure a =systemd= *timer* and *service* to run this on an interval.
The latter is usually provided to you if you used your *package manager* to install *borgmatic*.
* Conclusion
If you've checked *borg* and found it too much work to script, give *borgmatic* a try.
I've been using borgmatic for few weeks now with no issues at all.
I recently hooked it to a monitoring system so I will have a better view on when it runs, how much time each run takes.
Also, if any of my backups fail I get notified by email. I hope you enjoy *borg* and *borgmatic* as much as I am.

247
posts/backup/automating-borg.rst

@ -1,247 +0,0 @@
.. title: Automating Borg
.. date: 2020-02-02
.. slug: automating-borg
.. updated: 2020-02-02
.. status: published
.. tags: backup, borgbackup, borg, borgmatic
.. category: backup
.. authors: Elia el Lazkani
.. description: We've had a look at **Borg** before, let's find out how to automate it.
.. type: text
In the :doc:`previous blog post <borgbackup>`, I talked about **borg**.
If you read that post, you would've noticed that **borg** has a lot of features.
With a lot of features come a lot of automation.
If you were thinking about using **borg**, you should either make a *simple cron*
or you're gonna have to write an elaborate script to take care of all the different steps.
What if I told you there's another way ? An easier way ! The **Borgmatic** way...
What would you say ?
.. TEASER_END
Borgmatic
=========
**Borgmatic** is defined on their `website <https://torsion.org/borgmatic/>`_ as follows.
borgmatic is simple, configuration-driven backup software for servers and workstations.
Protect your files with client-side encryption. Backup your databases too.
Monitor it all with integrated third-party services.
If you go down to it, **borgmatic** uses **borg**'s *API* to automate a list of configurable *tasks*.
This way, it saves you the trouble of writing your own scripts to automate these steps.
**Borgmatic** uses a *YAML* configuration file. Let's configure a few tasks.
Location
========
First, let's start by configuring the locations that **borg** is going to be working with.
.. code:: yaml
location:
source_directories:
- /home/
repositories:
- user@backupserver:sourcehostname.borg
one_file_system: true
exclude_patterns:
- /home/*/.cache
- '*.pyc'
This tells **borg** that we need to backup our ``/home`` directories excluding a few patterns.
Let's not forget that we told **borg** where the repository is located at.
Storage
=======
We need to configure the storage next.
.. code:: yaml
storage:
# Recommended
# encryption_passcommand: secret-tool lookup borg-repository repo-name
encryption_passphrase: "ReallyStrongPassphrase"
compression: zstd,15
ssh_command: ssh -i /path/to/private/key
borg_security_directory: /path/to/base/config/security
archive_name_format: 'borgmatic-{hostname}-{now}'
In this section, we tell borg a little big of information about our repository.
What are the credentials, where it can find them, etc.
The easy way is to go with a ``passphrase``, but I recommend using an ``encryption_passcommand`` instead.
I also use ``zstd`` for encryption instead of ``lz4``, you better do your research before you change the default.
I also recommend, just as they do, the use of a security directory as well.
Retention
=========
We can configure a retention for our backups, if we like.
.. code:: yaml
retention:
keep_hourly: 7
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 2
prefix: "borgmatic-"
The part of what to keep from *hourly* to *daily* is self explanatory.
I would like to point out the ``prefix`` part as it is important.
This is the *prefix* that **borgmatic** uses to consider backups for **pruning**.
.. warning::
Watch out for the retention ``prefix``
Consistency
===========
After the updates, we'd like to check our backups.
.. code:: yaml
consistency:
checks:
- repository
- archives
check_last: 3
prefix: "borgmatic-"
.. warning::
Watch out, again, for the consistency ``prefix``
Hooks
=====
Finally, hooks.
I'm going to talk about hooks a bit. Hooks can be used to backup **MySQL**, **PostgreSQL** or **MariaDB**.
They can also be hooks for ``on_error``, ``before_backup``, ``after_backup``, ``before_everything`` and ``after_everything``.
You can also hook to third party services which you can check on their webpage.
I deployed my own, so I configured my own.
Borgmatic Configuration
=======================
Let's put everything together now.
.. code:: yaml
location:
source_directories:
- /home/
repositories:
- user@backupserver:sourcehostname.borg
one_file_system: true
exclude_patterns:
- /home/*/.cache
- '*.pyc'
storage:
# Recommended
# encryption_passcommand: secret-tool lookup borg-repository repo-name
encryption_passphrase: "ReallyStrongPassphrase"
compression: zstd,15
ssh_command: ssh -i /path/to/private/key
borg_security_directory: /path/to/base/config/security
archive_name_format: 'borgmatic-{hostname}-{now}'
retention:
keep_hourly: 7
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 2
prefix: "borgmatic-"
consistency:
checks:
- repository
- archives
check_last: 3
prefix: "borgmatic-"
Now that we have everything together, let's save it in ``/etc/borgmatic.d/home.yaml``.
Usage
=====
If you have **borg** and **borgmatic** already installed on your system and the **borgmatic** configuration file in place,
you can test it out.
You can create the repository.
.. code:: text
# borgmatic init -v 2
You can list the backups for the repository.
.. code:: text
# borgmatic list --last 5
borgmatic-home-2020-01-30T22:01:30 Thu, 2020-01-30 22:01:42 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-01-31T22:02:12 Fri, 2020-01-31 22:02:24 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-01T22:01:34 Sat, 2020-02-01 22:01:45 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-02T16:01:22 Sun, 2020-02-02 16:01:32 [0000000000000000000000000000000000000000000000000000000000000000]
borgmatic-home-2020-02-02T18:01:36 Sun, 2020-02-02 18:01:47 [0000000000000000000000000000000000000000000000000000000000000000]
You could run a check.
.. code:: text
# borgmatic check -v 1
/etc/borgmatic.d/home.yaml: Pinging Healthchecks start
/borg/home: Running consistency checks
Remote: Starting repository check
Remote: Starting repository index check
Remote: Completed repository check, no problems found.
Starting archive consistency check...
Analyzing archive borgmatic-home-2020-02-01T22:01:34 (1/3)
Analyzing archive borgmatic-home-2020-02-02T16:01:22 (2/3)
Analyzing archive borgmatic-home-2020-02-02T18:01:36 (3/3)
Orphaned objects check skipped (needs all archives checked).
Archive consistency check complete, no problems found.
summary:
/etc/borgmatic.d/home.yaml: Successfully ran configuration file
But most of all, if you simply run ``borgmatic`` without any parameters, it will run through the whole configuration and apply all the steps.
At this point, you can simply add the ``borgmatic`` command in a **cron** to run on an interval.
The other options would be to configure a ``systemd`` **timer** and **service** to run this on an interval.
The latter is usually provided to you if you used your **package manager** to install **borgmatic**.
Conclusion
==========
If you've checked **borg** and found it too much work to script, give **borgmatic** a try.
I've been using borgmatic for few weeks now with no issues at all.
I recently hooked it to a monitoring system so I will have a better view on when it runs, how much time each run takes.
Also, if any of my backups fail I get notified by email. I hope you enjoy **borg** and **borgmatic** as much as I am.

117
posts/backup/borgbackup.org

@ -0,0 +1,117 @@
#+BEGIN_COMMENT
.. title: BorgBackup
.. date: 2020-01-30
.. slug: borgbackup
.. updated: 2020-01-30
.. status: published
.. tags: backup, borgbackup, borg
.. category: backup
.. authors: Elia el Lazkani
.. description: It has been called the Holy Grail of backup! BorgBackup is coming to town.
.. type: text
#+END_COMMENT
I usually lurk around *Freenode* in a few projects that I use, can learn from and/or help with. This is a great opportunity to learn new things /all the time/.
This story is familiar in that manner, but that's where similarities diverge. Someone asked around =#Weechat= a question that caught my attention because it was, sort of, out of topic. The question was around how do you backup your stuff ?
{{{TEASER_END}}}
I mean if I were asked that, I would've mentioned revision controlled off-site repositories for the code that I have.
For the personal stuff on the other hand, I would've admitted simple rudimentary solutions like =rsync=, =tar= and external drives.
So I was sort of happy with my backup solution, it has worked. Plain and simple.
I have to admit that, by modern standards it might not offer the ability to go back in time to a certain point.
But I use /file systems/ that offer /snapshot/ capabilities. I can recover from previous snapshots and send them somewhere safe.
Archiving and encrypting those is not a simple process, wish it was. That limits storage possibilities if you care to keep your data private.
But if you know me, you'd know that I'm always open to new ways of doing things.
I can't remember exactly the conversation but the name *BorgBackup* was mentioned (thank you however you are). That's when things changed.
* BorgBackup
[[https://www.borgbackup.org/][Borg]] is defined as a
#+BEGIN_QUOTE
Deduplicating archiver with compression and encryption
#+END_QUOTE
Although this is a very accurate and encompassing definition, it doesn't really show you how /AWESOME/ this thing is.
I had to go to the docs first before I stumbled upon this video.
#+BEGIN_HTML
<div class="custom-center" style="
width: 50%;
margin: 0 auto;
">
<script id="asciicast-133292" src="https://asciinema.org/a/133292.js" async></script>
</div>
#+END_HTML
It can be a bit difficult to follow the video, I understand.
This is why I decided to write this post, to sort of explain to you how *Borg* can backup your stuff.
* Encryption
Oh yeah, that's the *first* thing I look at when I consider any suggested backup solution. *Borg* offers built-in /encryption/ and /authentication/. You can read about it in details in the [[https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-modes][docs]].
So that's a check.
* Compression
This is another thing I look for in a suggested backup solution. And I'm happy to report that *Borg* has this under the belt as well.
*Borg* currently supports /LZ4/, /zlib/, /LZMA/ and /zstd/. You can also tune the level of compression. Pretty neat !
* Full Backup
I've watched a few videos and read a bit of their documentation and they talk about *FULL BACKUP*.
Which means every time you run *Borg*, it will take a full backup of your stuff. A full backup at that point in time, don't forget.
The implication of this is that you have a versioned list of your backups, and you can go back in time to any of them.
Yes, you read that right. *Borg* does a full backup every time you run it. That's a pretty neat feature.
If you're a bit ahead of me, you were gonna say woooow there bud ! I have *Gigabytes* of data, what do you mean *FULL BACKUP*, you keep saying *FULL BACKUP*.
I mean *FULL BACKUP*, wait until you hear about the next feature.
* Deduplication
Booyah ! It has deduplication. Ain't that awesome. I've watched a presentation by the project's original maintainer explain this.
I have one thing to say. It's pretty good. How good, you may ask ?
My answer would be, good enough to fool me into thinking that it was taking snapshots of my data.
#+BEGIN_EXAMPLE
-----------------------------------------------------------------------------
Original size Compressed size Deduplicated size
All archives: 34.59 GB 9.63 GB 1.28 GB
Unique chunks Total chunks
Chunk index: 47772 469277
#+END_EXAMPLE
It wasn't until I dug in deeper into the matter that I understood that it was a full backup and the deduping taking care of the rest.
* Check
*Borg* offers a way to vefiry the consistency of the repository and the archives within. This way, you can make sure that your backups haven't been corrupted.
This is a very good feature, and a must in my opinion from a backup solution. *Borg* has /YOU/ covered.
* Restore
A backup solution is nothing if you can't get your data backup.
*Borg* has a few ways for you to get your data.
You can either create an /archive/ file out of a backup. You can export a file, a directory or the whole directory tree from a backup.
You can also, if you like, mount a backup and get stuff out.
#+BEGIN_EXPORT html
<div class="admonition warning">
<p class="admonition-title">warning</p>
#+END_EXPORT
Mounting a *Borg* backup is done using /fuse/
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
* Conclusion
*Borg* is a great tool for backup. It comes in an easily installable self-contained binary so you can use it, pretty much, anywhere giving you no excuse /whatsoever/ not to use it.
Their documentation is very good, and *Borg* is easy to use.
It offers you all the features you need to do off-site and on-site backups of all your important data.
I'll be testing *Borg* moving forward for my data. I'll make sure to report back anything I find, in the future, related to the subject.

131
posts/backup/borgbackup.rst

@ -1,131 +0,0 @@
.. title: BorgBackup
.. date: 2020-01-30
.. slug: borgbackup
.. updated: 2020-01-30
.. status: published
.. tags: backup, borgbackup, borg
.. category: backup
.. authors: Elia el Lazkani
.. description: It has been called the Holy Grail of backup! BorgBackup is coming to town.
.. type: text
I usually lurk around **Freenode** in a few projects that I use, can learn from and/or help with.
This is a great opportunity to learn new things *all the time*.
This story is familiar in that manner, but that's where similarities diverge.
Someone asked around ``#Weechat`` a question that caught my attention because it was, sort of, out of topic. The question was around how do you backup your stuff ?
.. TEASER_END
I mean if I were asked that, I would've mentioned revision controlled off-site repositories for the code that I have.
For the personal stuff on the other hand, I would've admitted simple rudimentary solutions like ``rsync``, ``tar`` and external drives.
So I was sort of happy with my backup solution, it has worked. Plain and simple.
I have to admit that, by modern standards it might not offer the ability to go back in time to a certain point.
But I use *file systems* that offer *snapshot* capabilities. I can recover from previous snapshots and send them somewhere safe.
Archiving and encrypting those is not a simple process, wish it was. That limits storage possibilities if you care to keep your data private.
But if you know me, you'd know that I'm always open to new ways of doing things.
I can't remember exactly the conversation but the name **BorgBackup** was mentioned (thank you however you are). That's when things changed.
BorgBackup
==========
`Borg <https://www.borgbackup.org/>`_ is defined as a
Deduplicating archiver with compression and encryption
Although this is a very accurate and encompassing definition, it doesn't really show you how *AWESOME* this thing is.
I had to go to the docs first before I stumbled upon this video.
.. raw:: html
<div class="custom-center" style="
width: 50%;
margin: 0 auto;
">
<script id="asciicast-133292" src="https://asciinema.org/a/133292.js" async></script>
</div>
It can be a bit difficult to follow the video, I understand.
This is why I decided to write this post, to sort of explain to you how **Borg** can backup your stuff.
Encryption
==========
Oh yeah, that's the **first** thing I look at when I consider any suggested backup solution.
**Borg** offers built-in *encryption* and *authentication*.
You can read about it in details in the `docs <https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-modes>`_.
So that's a check.
Compression
===========
This is another thing I look for in a suggested backup solution. And I'm happy to report that **Borg** has this under the belt as well.
**Borg** currently supports *LZ4*, *zlib*, *LZMA* and *zstd*. You can also tune the level of compression. Pretty neat !
Full Backup
===========
I've watched a few videos and read a bit of their documentation and they talk about **FULL BACKUP**.
Which means every time you run **Borg**, it will take a full backup of your stuff. A full backup at that point in time, don't forget.
The implication of this is that you have a versioned list of your backups, and you can go back in time to any of them.
Yes, you read that right. **Borg** does a full backup every time you run it. That's a pretty neat feature.
If you're a bit ahead of me, you were gonna say woooow there bud ! I have **Gigabytes** of data, what do you mean **FULL BACKUP**, you keep saying **FULL BACKUP**.
I mean **FULL BACKUP**, wait until you hear about the next feature.
Deduplication
=============
Booyah ! It has deduplication. Ain't that awesome. I've watched a presentation by the project's original maintainer explain this.
I have one thing to say. It's pretty good. How good, you may ask ?
My answer would be, good enough to fool me into thinking that it was taking snapshots of my data.
.. code:: text
-----------------------------------------------------------------------------
Original size Compressed size Deduplicated size
All archives: 34.59 GB 9.63 GB 1.28 GB
Unique chunks Total chunks
Chunk index: 47772 469277
It wasn't until I dug in deeper into the matter that I understood that it was a full backup and the deduping taking care of the rest.
Check
=====
**Borg** offers a way to vefiry the consistency of the repository and the archives within.
This way, you can make sure that your backups haven't been corrupted.
This is a very good feature, and a must in my opinion from a backup solution. **Borg** has *YOU* covered.
Restore
=======
A backup solution is nothing if you can't get your data backup.
**Borg** has a few ways for you to get your data. You can either create an *archive* file out of a backup.
You can export a file, a directory or the whole directory tree from a backup.
You can also, if you like, mount a backup and get stuff out.
.. warning::
Mounting a **Borg** backup is done using *fuse*
Conclusion
==========
**Borg** is a great tool for backup. It comes in an easily installable self-contained binary so you can use it, pretty much,
anywhere giving you no excuse *whatsoever* not to use it. Their documentation is very good, and **Borg** is easy to use.
It offers you all the features you need to do off-site and on-site backups of all your important data.
I'll be testing **Borg** moving forward for my data. I'll make sure to report back anything I find, in the future, related to the subject.

480
posts/configuration-management/ansible-testing-with-molecule.org

@ -0,0 +1,480 @@
#+BEGIN_COMMENT
.. title: Ansible testing with Molecule
.. date: 2019-01-11
.. slug: ansible-testing-with-molecule
.. updated: 2019-06-21
.. status: published
.. tags: configuration management, ansible, molecule,
.. category: configuration management
.. authors: Elia el Lazkani
.. description: A fast way to create a testable ansible role using molecule.
.. type: text
#+END_COMMENT
When I first started using [[https://www.ansible.com/][ansible]], I did not know about [[https://molecule.readthedocs.io/en/latest/][molecule]]. It was a bit daunting to start a /role/ from scratch and trying to develop it without having the ability to test it. Then a co-worker of mine told me about molecule and everything changed.
{{{TEASER_END}}}
I do not have any of the tools I need installed on this machine, so I will go through, step by step, how I set up ansible and molecule on any new machine I come across for writing ansible roles.
* Requirements
What we are trying to achieve in this post, is a working ansible role that can be tested inside a docker container. To be able to achieve that, we need to install docker on the system. Follow the instructions on [[https://docs.docker.com/install/][installing docker]] found on the docker website.
* Good Practices
First thing's first. Let's start by making sure that we have python installed properly on the system.
#+BEGIN_EXAMPLE
$ python --version
Python 3.7.1
#+END_EXAMPLE
Because in this case I have /python3/ installed, I can create a /virtualenv/ easier without the use of external tools.
#+BEGIN_EXAMPLE
# Create the directory to work with
$ mkdir -p sandbox/test-roles
# Navigate to the directory
$ cd sandbox/test-roles/
# Create the virtualenv
~/sandbox/test-roles $ python -m venv .ansible-venv
# Activate the virtualenv
~/sandbox/test-roles $ source .ansible-venv/bin/activate
# Check that your virtualenv activated properly
(.ansible-venv) ~/sandbox/test-roles $ which python
/home/elijah/sandbox/test-roles/.ansible-venv/bin/python
#+END_EXAMPLE
At this point, we can install the required dependencies.
#+BEGIN_EXAMPLE
$ pip install ansible molecule docker
Collecting ansible
Downloading https://files.pythonhosted.org/packages/56/fb/b661ae256c5e4a5c42859860f59f9a1a0b82fbc481306b30e3c5159d519d/ansible-2.7.5.tar.gz (11.8MB)
100% |████████████████████████████████| 11.8MB 3.8MB/s
Collecting molecule
Downloading https://files.pythonhosted.org/packages/84/97/e5764079cb7942d0fa68b832cb9948274abb42b72d9b7fe4a214e7943786/molecule-2.19.0-py3-none-any.whl (180kB)
100% |████████████████████████████████| 184kB 2.2MB/s
...
Successfully built ansible ansible-lint anyconfig cerberus psutil click-completion tabulate tree-format pathspec future pycparser arrow
Installing collected packages: MarkupSafe, jinja2, PyYAML, six, pycparser, cffi, pynacl, idna, asn1crypto, cryptography, bcrypt, paramiko, ansible, pbr, git-url-parse, monotonic, fasteners, click, colorama, sh, python-gilt, ansible-lint, pathspec, yamllint, anyconfig, cerberus, psutil, more-itertools, py, attrs, pluggy, atomicwrites, pytest, testinfra, ptyprocess, pexpect, click-completion, tabulate, future, chardet, binaryornot, poyo, urllib3, certifi, requests, python-dateutil, arrow, jinja2-time, whichcraft, cookiecutter, tree-format, molecule, docker-pycreds, websocket-client, docker
Successfully installed MarkupSafe-1.1.0 PyYAML-3.13 ansible-2.7.5 ansible-lint-3.4.23 anyconfig-0.9.7 arrow-0.13.0 asn1crypto-0.24.0 atomicwrites-1.2.1 attrs-18.2.0 bcrypt-3.1.5 binaryornot-0.4.4 cerberus-1.2 certifi-2018.11.29 cffi-1.11.5 chardet-3.0.4 click-6.7 click-completion-0.3.1 colorama-0.3.9 cookiecutter-1.6.0 cryptography-2.4.2 docker-3.7.0 docker-pycreds-0.4.0 fasteners-0.14.1 future-0.17.1 git-url-parse-1.1.0 idna-2.8 jinja2-2.10 jinja2-time-0.2.0 molecule-2.19.0 monotonic-1.5 more-itertools-5.0.0 paramiko-2.4.2 pathspec-0.5.9 pbr-4.1.0 pexpect-4.6.0 pluggy-0.8.1 poyo-0.4.2 psutil-5.4.6 ptyprocess-0.6.0 py-1.7.0 pycparser-2.19 pynacl-1.3.0 pytest-4.1.0 python-dateutil-2.7.5 python-gilt-1.2.1 requests-2.21.0 sh-1.12.14 six-1.11.0 tabulate-0.8.2 testinfra-1.16.0 tree-format-0.1.2 urllib3-1.24.1 websocket-client-0.54.0 whichcraft-0.5.2 yamllint-1.11.1
#+END_EXAMPLE
* Creating your first ansible role
Once all the steps above are complete, we can start by creating our first ansible role.
#+BEGIN_EXAMPLE
$ molecule init role -r example-role
--> Initializing new role example-role...
Initialized role in /home/elijah/sandbox/test-roles/example-role successfully.
$ tree example-role/
example-role/
├── defaults
│ └── main.yml
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── molecule
│ └── default
│ ├── Dockerfile.j2
│ ├── INSTALL.rst
│ ├── molecule.yml
│ ├── playbook.yml
│ └── tests
│ ├── __pycache__
│ │ └── test_default.cpython-37.pyc
│ └── test_default.py
├── README.md
├── tasks
│ └── main.yml
└── vars
└── main.yml
9 directories, 12 files
#+END_EXAMPLE
You can find what each directory is for and how ansible works by visiting [[https://docs.ansible.com][docs.ansible.com]].
** =meta/main.yml=
The meta file needs to modified and filled with information about the role. This is not a required file to modify if you are keeping this for yourself, for example. But it is a good idea to have as much information as possible if this is going to be released. In my case, I don't need any fanciness as this is just sample code.
#+BEGIN_SRC yaml
---
galaxy_info:
author: Elia el Lazkani
description: This is an example ansible role to showcase molecule at work
license: license (BDS-2)
min_ansible_version: 2.7
galaxy_tags: []
dependencies: []
#+END_SRC
** =tasks/main.yml=
This is where the magic is set in motion. Tasks are the smallest entities in a role that do small and idempotent actions. Let's write a few simple tasks to create a user and install a service.
#+BEGIN_SRC yaml
---
# Create the user example
- name: Create 'example' user
user:
name: example
comment: Example user
shell: /bin/bash
state: present
create_home: yes
home: /home/example
# Install nginx
- name: Install nginx
apt:
name: nginx
state: present
update_cache: yes
notify: Restart nginx
#+END_SRC
** =handlers/main.yml=
If you noticed, we are notifying a handler to be called after installing /nginx/. All handlers notified will run after all the tasks complete and each handler will only run once. This is a good way to make sure that you don't restart /nginx/ multiple times if you call the handler more than once.
#+BEGIN_SRC yaml
---
# Handler to restart nginx
- name: Restart nginx
service:
name: nginx
state: restarted
#+END_SRC
** =molecule/default/molecule.yml=
It's time to configure molecule to do what we need. We need to start an ubuntu docker container, so we need to specify that in the molecule YAML file. All we need to do is change the image line to specify that we want an =ubuntu:bionic= image.
#+BEGIN_SRC yaml
---
dependency:
name: galaxy
driver:
name: docker
lint:
name: yamllint
platforms:
- name: instance
image: ubuntu:bionic
provisioner:
name: ansible
lint:
name: ansible-lint
scenario:
name: default
verifier:
name: testinfra
lint:
name: flake8
#+END_SRC
** =molecule/default/playbook.yml=
This is the playbook that molecule will run. Make sure that you have all the steps that you need here. I will keep this as is.
#+BEGIN_SRC yaml