Personal notes and occasional posts

2019 has been a great year for Linux.Pizza, we have seen hundreds of new users on our Mastodon-instance, thousands of visitors on our trashmail service and hundreds of thousand of machines using our mirror

but sadly, due to lack of time and money, I have been forced to close down a few service like Pleroma, Social relay, & Invidious.

Linux.Pizza is one of many providers in the librehost network – everyone aims to offer ethical services that focus on privacy and fairness. 2019 has been a year of growth for the network and Linux.Pizza is not excluded.

Linux.Pizza was able to achieve the following:

2020 is approaching rapidly, and Linux.Pizza is ready! If you have any suggestions of services that you would like to see, please let us know! We are always exploring ways extend our list of services that is beneficial to our users!

In your haproxy.cfg, you should do this:

    log /dev/log local0 notice
    user haproxy
    group haproxy
    stats socket /var/run/haproxy.sock mode 660 level admin
    log global
    retries 2
    timeout connect 3000
    timeout server 5000
    timeout client 5000
listen galera
    bind /var/run/mysqld/mysqld.sock mode 666
    option mysql-check user HAPROXY
    balance first
    server galera1 check maxconn 64
    server galera2 check maxconn 64
    server galera3 check maxconn 64

And why do I listen on a socket? That is because an application will assume that mysql/mariadb are listening on a socket when you specify “localhost” by default.

This assumes that you are running your Linuxsystem as a virtual server on VMWare, KVM, XEN or AHV.

We have been there a couple of times – your application need more RAM. Or just want more RAM in order to have your Minecraft server run smoothly.

  1. First of all, add the amount of RAM to your VM via your hypervisor.
  2. Second, we need to tell the kernel that there is more RAM available for use, we can do that by using this simple script:

for f in /sys/devices/system/memory/memory*/state ; do grep -v -q online $f || continue echo -n "Bringing $f online... " echo online > $f || continue echo OK done

That's all! You can now verify with your favorite way to check RAM usage.

This assumes that you are running your Linuxsystem as a virtual server on VMWare, KVM, XEN or AHV.

Be sure to take a backup first!

It is quite common – for me atleast – that a busy databaseserver is going to use more and more space. And in most cases it is not always that attractive to have to restart the server and resize it with a LiveCD such as gparted live.

Luckily, there is possible to expand the partitions while your machine is running. I will try to guide you with this step to step guide on how you can successfully proceed with it:

If you are using LVM, read until 12, and skip 13.

  1. First of all, add the storage in you hypervisor.
  2. Next, you want to tell the kernel that to rescan the bulk device: echo 1 > /sys/class/block/sda/device/rescan fdisk are now able to “see” the expanded disk, in this case /dev/sda
  3. Now, we need to do the actual expansion of the partinion, so we need to “delete” the partion and add it again via fdisk – dont worry, we are not going to write the changes to disk while the partinion is deleted: fdisk /dev/sda
  4. Just to make sure, check the partinion by selecting p, if the disk looks like it should, you may continue
  5. Delete the partition by pressing d
  6. Press n to create a new partinion
  7. Press p to chose “Primary Partition”
  8. Press 1 to chose the partition number 1
  9. Press ENTERtwice
  10. And lastly, doublecheck the changes by pressing p, note the difference from the first time you did it. You should see that the partition has been expanded.
  11. !The following actions will write the changes to disk! Now, in order to make is real, press w in order to write the changes to disk, you will see a warning about that the partition table has been changed. Do not worry, it is expected.
  12. Now, run partprobe (you have to install parted for this to work). If you are using an older kernel, you maybe would need to reboot if the partprobe task did not work.
  13. Now, we just need to expand the filesystem itself in order to actually use the newly added space: resize2fs -p /dev/sda1

Verify with df -h or lsblk

Steps for if you are using LVM

I assume that you have done step 1-12 in the previous section.

  1. First, expand your physical volume: pvresize /dev/sda1
  2. Then, we will proceed with expanding the Logical volume, replace “X1” with the Logical Volume you want to extend: lvextend --extents +100%FREE /dev/vg0/X1
  3. After that is done, we have to expand the filesystem, again – replace “X1” with the Logical Volume you want to extend resize2fs /dev/vg0/X1

Verify with df -h or lsblk

You should be able to see the new size now.

This assumes that you are running your Linuxsystem as a virtual server on VMWare, KVM, XEN or AHV with CPU Hotplug enabled.

From time to time, I have been in need of extra CPU-power for various reasons. It might be that someone suddenly got alot of traffic to their PHP-based website, or they want to add more containers but loosing the time and uptime that comes with a reboot it not an option.

Luckily, there is solutions for that!

  1. First of all, add the cores to your VM via your hypervisor.
  2. Second, we need to tell the kernel that there is more cores available for use, we can do that by using this simple script:

for f in /sys/devices/system/cpu/cpu*/online ; do grep -v -q 0 $f || continue echo -n "Bringing $f online... " echo 1 > $f || continue echo OK done

Done, you can now verify that you have added more cores to your system, using for example – htop

Note! Some programs needs a restart in order to start using more cores

It is almost 18 years since I first came in contact with Linux, and I did not know at all what kind of journey it would take me on.

I was 9 years old at the time, when me and my brother got our first PC. It was an Fujitsu-Siemens with 128MB of RAM, Pentium II 233 MHZ and a HDD with amazing 8GB space. SuSE 7.0 was installed on that machine and we loved it. Although at the time, we only used Linux so we can brag to our friends that “we didn't use Windows”. That part kind of failed because our friends did'nt even know what Windows was and my brother and I thought that was absurd that our friends was so “informed about something so essential in life”.

At school, I loved to just “click around” and find stuff. Schools in the 90's/early 2000 did not consider “security” as something prioritized = more fun for us kids.

Today, I mainly work with Linux and basically almost every hobby that I have has something to do with Linux.

It is amazing how such a small event as getting a PC with Linux could alter the course of my life.

Why I decided to block gab.[com|ai], and some other thoughts.

It has not gone unoticed, but gab – the plattform that champions “free speechs” has joined the fediverse. Why? Because probably (hopefully) they are running out of money.

4th of July, 2019 – gab did enter the fediverse with glory and might. Users loved it and the software was great!

gab-5 No it was'nt. People hated it and complained everywhere over the extremely bad performance and the unreliable service. I enjoyed the show, it was great because it means that gab is loosing money when the deploy did not work well. gab-500 gab also uses cloudflare gab-cf


The funny thing aside for a moment, and lets focus on the dangerous and evil side of gab. Gab claims to be a champion of free speech, sadly this has turned into a harbor of extremist – actual Nazis.

I decided to create an account on the platform to observe it myself. I did not have to scroll much to be greeted with this: gab-1

More: gab-2

And more: gab-3 This was presented to me withing the first minute.

gab-trending So much for the All are welcome slogan.

You maybe wonder – why does a platform allow content that celebrates genocide, fascism , violence, hate, and authoritarian governments and hides it in the free speech bucket? I do not know, I honestly do not know. The founder of gab – Andrew Torba – is even a proclaimed christian. Is'nt christianity supposed to be the opposite of the content that is spreading on gab?

This is why I have suspended gab on my instance. And if you as an admin does it – great! But it would also be great if people could federate with them to be able to ridicule them (or as some people calls it – virtual milk-shaking).

We can't let history repeat itself.

Note: I do publish's blocklist here

Since I was a little kid I've always had a love for computes. Later on, that love transformed to a love for servers – hence that's why I got myself a tiny homelab.

Currently, the homelab is rocking the following setup: One Proliant Microserver G8 8TB disk, 8GB RAM, simple celeron CPU, 10Gbit Network running Debian 9. Currently using this as a shared storage (iSCSI) between my to hypervisors. I had two of those before, but I gave one away.

One homebuild server with an SuperMicro X9SCI/X9SCA motherboard, Xeon E3-1240 V2, 32GB RAM and 4TB storage (mostly ZFS with alot of SSD cache) Currently running Proxmox in a HA-environment

One homebuild PC that was repurposed into a server. ASRock H61 Pro BTC motherboard, Intel i5-2400, 16GB RAM and 1,1 TB of ZFS with SSD-cache storage. Also running Proxmox in a HA-environment

An EdgeRouter X-SFP (serving the secondary connection”

Three MikroTik Switches – RouterBOARD 260GS, 260GSP & CSS326-24G-2S+RM.

A Meraki accesspoint

Thinkpad W530

This is the hardware my environment utilizes. I am running Pfsense in a virtual environment instead of using the Edgerouter as the main router. The reason is that I wanted a more advanced firewall than the EdgeRouter on my “main” connection. And the reason that I run pfsense in a virtual machine is because I want the ability to easily restore backups in case if something goes wrong – which pfsense often do when I upgrade for some reason.

I hope that I can get a newer server in the coming month so I can replace my ASRock “server” and convert that one to a router instead – hopefully.

You probably came here because some friend of you mentioned “fediverse” to you, while you responded – fediverse? what is that?.

And your friend did not really wanted to explain it, so you was sent here instead.

Great! I will try my best into explaining what the “Fediverse” is, and how it might save your digital life.

The Fediverse is a name of a network of tens of thousands connected servers, consisting of many different kind of software that “speaks” to each other. The most popular software is Mastodon, that basically is a Twitter clone. The big difference is that the code that is being used on the server is publicly available to everyone, so that ensures that the code is secure and does not hide any hidden “feature” such as data collection on a massive scale, such as Twitter.

Mastodon is developed by many, but the leader and founder of the project is Eugen “Gargron” Rochko that wanted to create a social network because he was tired of having his friends and family being on different networks. And if he wanted to keep in touch with he had to create account on different places (MySpace, Twitter, Viber, Telegram and so on). And so Mastodon was born.

Today, there is thousands of Mastodon servers up and running, each one with it's own characteristic, culture, rules and people. Servers for those who love beer, cats, politics, Linux or just being social. Most of the servers is connected to each other. Yes, you can join that super-niche server that focuses exclusively on cats with moustaches, while still connecting to your friends who is on that instance about helicopters – is'nt that awesome?

Or, if you just want to listen to someone explain how it works, here is a Peertube (which is also a fediverse compatible piece of software) video explaining how it works

As a MTA lover, I always try to encourage people (especially “IT-people”) to host their own mailserver. Mostly so they actually can learn something and also that I do not like how the big providers like Google, Microsoft, Amazon etc keep eating up the market. Diversity is a key to a healthy market – but that is another topic.

This guide will mostly apply to Debian-based distros like Debian (9 or newer), Ubuntu (16.04 or newer) or any other “serverdistro”. I do assume that you already have a working mailserver that do both deliver and receive emails that are DKIM signed (or atleast perform validation with OpenDKIM), otherwhise you can read my short guide here (coming soon).

First, install OpenDMARC from the repository.

apt update
apt install opendmarc -y

Verify that the user and group opendmarc has been created by checking /etc/passwd and /etc/group. Otherwhise, create them. When you have installed it, verify the installation by running this:

opendmarc -V

You will get something like this (the version number is not that important yet):

opendmarc: OpenDMARC Filter v1.3.2       
    SMFI_VERSION 0x1000001       
    libmilter version 1.0.1       
    Active code options:               

Great! Let's proceed to configuring opendmarc First, take a backup of the current opendmarc.conf, it will save some headache in the future if you want to redo it:

cp /etc/opendmarc.conf /etc/opendmarc.conf.BAK

Edit /etc/opendmarc.conf with the following:

FailureReports true
PidFile /var/run/
RejectFailures false
SPFSelfValidate yes
Socket inet:8893@localhost
SoftwareHeader true
Syslog true
SyslogFacility mail
HistoryFile /var/run/opendmarc/opendmarc.dat
UMask 0002
UserID opendmarc

Dont forget to restart opendmarc

service opendmarc restart 

Proceed with adding opendmarc as a milter in postfix. I am assuming that you already have opendkim enabled as a milter like this:

smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

We now need to add the opendmarc milter into the postfix configuration, it is important that you add it AFTER the opendkim milter, otherwhise opendmarc will not be able to check if the DKIM key is valid.

smtpd_milters = inet:localhost:8891,inet:localhost:8893
non_smtpd_milters = inet:localhost:8891,inet:localhost:8893
milter_default_action = accept

The last one is pretty important, so if one of your milters does not work for some reason – Postfix will still let it throu. Restart postfix

service postfix restart

We should now be able to test the configuration by sending an email from example a account to an email address on your email-server and check your logs if opendmarc actually works.

tail -f /var/log/mail.log | grep "opendmarc"

You should be able to see this:

Apr 26 12:16:38 mx opendmarc[31490]: 5155751C32: SPF(mailfrom): pass
Apr 26 12:16:39 mx opendmarc[31490]: 5155751C32: pass

Great! Your server does now validate DMARC policies! If you just wanted this basic functionality, you are done now. But there is always room for improvement!

Adding a Public-suffix list

This can be achieved in the following simple steps: Create a catalogue (and change ownership) for the list to be downloaded to:

mkdir -p /etc/opendmarc/
chown opendmarc: /etc/opendmarc

Set up a cronjob to download the suffix list once a week

crontab -u opendmarc -e

And this line:

@weekly/usr/bin/wget -k -q -N -P /etc/opendmarc

Also, just download the list so you have it before you configure opendmarc to use it:

wget -k -q -N -P /etc/opendmarc 

Finally, configure opendmarc to actually use that list, put this on the bottom in /etc/opendmarc.conf and restart opendmarc

PublicSuffixList /etc/opendmarc/effective_tld_names.dat
service opendmarc restart 

Awesome! You are now done with the OpenDMARC. Next up – adding DMARC reporting, this will be in an upcoming post.

#postfix #dmarc #opendmarc #smtp #email #linux