LinuxPizza

Personal notes and occasional posts

Hopefully this will save some of you alot of time, energy, and save you day.

I recently had troubles getting a job to work. The short story is:

Download all files in a remote catalogue, over SFTP, on certain times.

I had a working solution with curl, but when the naming of the files changed (such as whitespaces) – the function broke.

lftp – the saver

After have spent a couple of hours trying to grasp lftp via the manpage, I came up with a solution:

lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'

And if you want to remove the source-files after download:

lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --Remove-source-files --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'

This download all files in the specified remote catalogue to the specified local one, then exits.

#linux #bash #sftp #lftp

Here is a post about Windows for a change.

If you want to check if you can query a NTP-server from your Windows-machine, you can just use the following

w32tm /stripchart /computer:computername

For example:

w32tm /stripchart /computer:ntp.netnod.se

If everything works, you'll see something like this:

Tracking ntp.netnod.se [194.58.200.20:123].
The current time is 2022-12-06 14:06:13.
14:06:13, d:+00.0260863s o:+00.0277480s  [      *      ]

Have a pleasant tuesday

#windows #ntp

OmniOS - The distant cousin to Linux

Linux distant cousin – OmniOS

Not so long ago, there where plenty of cousins alive and well in the world. You may have heard about Solaris, IRIX, AIX, HP-UX – in this post, we will explore the grand-child to SunOS called OmniOS

OmniOS is based on the Illumos kernel – a fork of the long forgotten and abandoned OpenSolaris kernel that famously was killed by Oracle shortly after the takeover from Sun Microsystems (rest in peace).

I personally, are not fond of a homogeneous IT world, where every system is basically the same. I do think that diversity is good. So I hope that I can you as a reader interested in different Linux-alternatives (that is not BSD).

So, pick the hypervisor you want (Proxmox, VMWare, AHV, VirtualBox) and navigate to https://omnios.org/download to download the latest stable .iso.

The installation itself is very straight forward, so I'll be skipping that part here.

When you have installed the system, and rebooted – we need to get networking up and running. Login to the machine with the user “root”, and no password.

Lets enable networking, start by displaying the network-ports on your system. In my case – I'm using a virtualized Intel e1000 network card (available in basically every hypervisor)

dladm show-link -o link

Output:

LINK
e1000g0

Great, since this virtual machine only have a single NIC – this output is expected.

Lets create the interface, give it an IPv4 and add a default route via the network gateway:

ipadm create-if e1000g0
ipadm create-addr -T static -a 192.168.2.38/24 e1000g0/v4
route -p add default 192.168.2.254

Verify that you have network connection with ping:

ping 8.8.8.8
8.8.8.8 is alive

DNS-resolution is not yet working, so we have to take care of that. Lets modify /etc/resolv.conf with the following

echo "domain local.lan" > /etc/resolv.conf
echo "nameserver 192.168.1.254" >> /etc/resolv.conf

In some cases, I have had to overwrite nsswitch.conf aswell:

cat /etc/nsswitch.dns > /etc/nsswitch.conf

We should be able to resolve our way into the internet now:

ping linux.pizza
linux.pizza is alive

Create a user and enable SSH, most of you will probably recognize the steps, since they are fairly basic and do not differ that much from how you would to in on a modern Linux machine

Let's start by create a user:

useradd -m -d /home/jonathan -s /bin/bash jonathan

And set a password for the user, you can do it for the root-user aswell:

passwd jonathan
New password:
...

And last, enable the SSH-service:

svcadm enable ssh  

Great, you should now be able to SSH-into the machine with the user you just created. You can elevate yourself into the root-user with the “su” command. Not that complicated, right?

Let's update the system.

Lets update the package-information from the repositories

pkg refresh
pkg update

Just let this run, your system will be updated. In some cases, your system will have to reboot.

And here, is where OmniOS shines – you can reboot your system – test it after you have performed the updates, and if stuff does not really work, you can simply rollback the changes you've made with beadm

Rollback the system

After you have rebooted your system, check the output of beadm list

root@omnios:~# beadm list
BE               Active Mountpoint Space  Policy Created
omnios-r151038an -      -          65.40M static 2022-12-03 15:35
omnios-r151038ca NR     /          1.35G  static 2022-12-03 16:14

You'll see to environments, and the one marked with “NR” is the current active one. If you want to rollback, you could execute (in my case) `beadm activate omnios-r151038an

root@omnios:~# beadm activate omnios-r151038an
Activated successfully
root@omnios:~# 
root@omnios:~# 
root@omnios:~# 
root@omnios:~# ls
1
root@omnios:~# beadm list
BE               Active Mountpoint Space   Policy Created
omnios-r151038an R      -          869.15M static 2022-12-03 15:35
omnios-r151038ca N      /          580.90M static 2022-12-03 16:14

Then do a quick reboot to boot into the active one. Let's check the current active environment after the system has rebooted:

root@omnios:~# beadm list
BE               Active Mountpoint Space   Policy Created
omnios-r151038an NR     /          872.95M static 2022-12-03 15:35
omnios-r151038ca -      -          583.64M static 2022-12-03 16:14

And we can see that the old one, created at 15:35 is the one that is currently active. We can now remove the newer one, since we dont need it anymore:

root@omnios:~# beadm destroy omnios-r151038ca
Are you sure you want to destroy omnios-r151038ca?
This action cannot be undone (y/[n]): y
Destroyed successfully
root@omnios:~# beadm list 
BE               Active Mountpoint Space   Policy Created
omnios-r151038an NR     /          748.05M static 2022-12-03 15:35

So, what is the practical / real world use for OmniOS?

Since OmniOS use ZFS, anything storage-related makes sense, such as a Network Attached Storage (NAS) even a Virtual Machine Host with the bhyve hypervisor – which itself is very interesting.

If you are interested in more, you can check out my older post about Tribblix here.

#unix #solaris #omnisos #illumos #tribblix #zfs

Have you ever installed packages from third party repositories to later realize that it was not the best idea? No? Well – I have.

But from now on, I wont do it. And instead utilize chroot!

In this section, we'll cover how you do it on your Debian-based system aswell as on Fedora.

Install the package debootstrap

apt update; apt install debootstrap

Or if you are on Fedora:

dnf install debootstrap

Now, create a catalogue anywhere of your choosing, for example in /srv

mkdir /srv/chroot

Great. Now depending on what you want. Perform any of the following:

debootstrap bullseye /srv/chroot http://ftp.us.debian.org/debian
debootstrap  jammy /srv/chroot http://archive.ubuntu.com/ubuntu

You should see it pull down all the packages needed for the distro to start, and when it is complete – you can enter the chroot:

chroot /srv/chroot

That's it! Now you can install, test or compile your packages as usual.

And when you are done, you can simply remove the catalogue and start fresh – if you want.

Here is a quick demo of the actual process:

asciicast

#linux #chroot #sysadmin #debian #ubuntu #fedora

Personal note, I always forget

docker stop $(docker ps -aq) && docker rm $(docker ps -aq)

Also run a docker system prune -a afterwards.

Now I'll remember it.

#linux #docker

This short writeup will guide you on how to do it on Debian-based and Fedora systems.

We begin with installing posfix and the required packages for authentication.

First – doublecheck that your machine has a Fully Qualified Domain Name set in the hostfile, this will remove alot of headaches from you in the future.

My machine is named “T15.domain.tld” – so emails will be arriving from “user@T15.domain.tld”.

Debian:

apt-get install postfix mailutils libsasl2-2

Fedora:

dnf install postfix mailx mailx cyrus-sasl cyrus-sasl-plain

Next, we will create the sasl-password file and hash it:

echo "[relay.domain.tld]:587 username:password" > /etc/postfix/sasl
postmap /etc/postfix/sasl

Great, now we have to tell postfix that all emails sent via it should be relayed throu the smtp-relay:

relayhost = [relay.domain.tld]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

Restart postfix and check the log, so everything looks good.

Test your setup:

echo "This is a test" | mail -s "Just a test" destination_email@domain.tld

You can check the status of the mailqueue with the command mailq.

And that's about it!

#linux #postfix #smtp #sysadmin #email

Personal note with example IPv4s Maybe this could be useful for your DNS-server aswell


IPT="/sbin/iptables"

# Flush old rules, old custom tables
$IPT --flush
$IPT --delete-chain

# Set default policies for all three default chains
$IPT -P INPUT DROP
$IPT -P FORWARD DROP
$IPT -P OUTPUT ACCEPT

# Enable free use of loopback interfaces
$IPT -A INPUT -i lo -j ACCEPT
$IPT -A OUTPUT -o lo -j ACCEPT

# All TCP sessions should begin with SYN
$IPT -A INPUT -p tcp ! --syn -m state --state NEW -s 0.0.0.0/0 -j DROP


# Accept inbound TCP packets

$IPT -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Traffic from Ports
$IPT -A INPUT -p tcp --dport 22 -m state --state NEW -s 192.168.69.0/24 -j ACCEPT
$IPT -A INPUT -p udp -m udp --dport 53 -s 192.168.69.X/32 -j ACCEPT
$IPT -A INPUT -p tcp --dport 53 -m state --state NEW -s 192.168.69.X/32 -j ACCEPT
$IPT -A INPUT -p tcp --dport 5666 -m state --state NEW -s 192.168.69.X/32 -j ACCEPT

# Flood protection for DNS UDP
# See: /proc/net/ipt_hashlimit/DNS
#iptables -A INPUT -p udp --dport 53 -m state --state NEW \
#  -m hashlimit \
#  --hashlimit-above 300/sec \
#  --hashlimit-mode srcip \
#  --hashlimit-name DNS \
#  --hashlimit-htable-expire 60000 \
#  -j DROP

$IPT -A INPUT -p tcp --dport 53 -m state --state NEW -s 0.0.0.0/0 -j ACCEPT  # - DNS


# Accept inbound UDP packets

$IPT -A INPUT -p udp -m udp --dport 53 -s 0.0.0.0/0 -j ACCEPT


# Accept inbound ICMP packets #

$IPT -A INPUT -p ICMP --icmp-type 8 -s 0.0.0.0/0 -j ACCEPT

There is many reasons why you would like to have local docker registry, and being cool is one of them. This is how you do it on Debian.

Well, lets go!

1 Install the package:

apt install docker-registry

2 Tell the machine where you are going to pull to, that the registry is “unsafe” (non-https):

{
  "insecure-registries":
    [
      "docker.internal:5000",
      "registry.linux.pizza:5000"
    ]
}

3 Create a basic-auth on your registry:

   htpasswd -Bbn dockeruser registrypassword >/etc/docker/registry/.htpasswd

4 Login to the registry from your host:

   docker login -u dockeruser registry.linux.pizza:5000

Dont forget to restart docker!

Have fun!

#docker #dockerregistry #registry #debian #linux

This fall, has been busy for me, and when I am busy I like to experiment and play with stuff – especially IT and nerdy stuff.

This brings me back to the days when I started with Linux, for me it was new, cool and different. Nowadays I feel like it is not so different.

So, being the hipster that I am, I set out to find something new. I've already tried and played with OmniOS a couple of years back and I liked it. And during my quest to get it on a iPXE server – I found out about Tribblix.

Tribblix is a Illumos distribution with a retro feel. And that I liked. The iPXE setup was easy since Peter Tribble – the creator and maintainer of Tribblix – already have an iPXE server up and running. So I just used the already ready target at (http://pkgs.tribblix.org/m24/ipxe.txt) and got it up and running according to the installation instructions provided by Tribblix here: http://www.tribblix.org/install.html

Enter Tor – and the recent news that Russia has decided to block Tor nationwide. I must say that I do understand why they did it from a IT-security perspective – alot of crap is originating from Russia. That is an undisputed fact. However, from a freedom perspective – this is making an already bad situation even worse. So what a perfect opporunity to run a Tor-relay on a SunOS machine! Currently, there is only 5 relays that is running on a Solaris/SunOS based kernel (including one of mine). Lets change that – more diversity in the Tor-network is a good thing.

So, assuming that you have installed Tribblix. Lets pull down the Tribblix overlay called “develop”

zap install develop

This takes a couple of minutes, and what it does is that it download and installs all the nessecary packages you need to build Tor.

We also need LibEvent, since Tor requires it. And the TRIBlibev is like libevent – but not really – we need to manually compile it.

Download the latest stable version from github, and extract it like shown below:

wget https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/libevent-2.1.12-stable.tar.gz
tar -xvf libevent-2.1.12-stable.tar.gz

Lets also pull down the Tor-source code and extract:

wget https://dist.torproject.org/tor-0.4.6.9.tar.gz
tar -xvf tor-0.4.6.9.tar.gz

Great, now you should have everything you need in order to get started.

Navigate to the libevent catalogue you just extracted, and run the following:

./configure --prefix=/tmp/mc --enable-static --disable-shared
gmake
gmake install

This will create a static and temporary copy of LibEvent in the /tmp/mc catalogue. we will use that in the compilation of Tor.

When the compilation of LibEvent is done, nagivate to the Tor-catalogue and run the following:

./configure --with-libevent-dir=/tmp/mc MAKE="gmake"
gmake
gmake install

The proceedure is quite similar to the LibEvent compilation.

When the gmake install procedure is done, you can find the tor-config files in /usr/local/etc/tor/ and the binaries in /usr/local/bin/.

Configure the torrc file according to your needs, read the Tor Projects Post Install Guide and follow the recommendations.

You can now run tor by just typing /usr/local/bin/tor in your terminal – and you now have a relay running.

For now, you can use tmux in order to run it in the background.

Thanks to Peter Tribble, who told me how to get LibEvent into the system.

Tor Relay by Kernels

#unix #tor #building #illumos #tribblix

Took myself ages to figure this out, so I am noting this down for my future self. Just a note – this is not the indented workflow, but rather a “getting started with kubernetes” step.

First, we need to add NFS as a storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

Then, we can add the actual storage:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: nfs-persistentvolume
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  storageClassName: "nfs" # Empty string must be explicitly set otherwise default StorageClass will be set / or custom storageClassName name
  nfs:
    path: "/path/to/share"
    server: "xxx.xxx.xxx.xxx"
    readOnly: false
  claimRef:
    name: nfs-persistentvolumeclaim
    namespace: default
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-persistentvolumeclaim
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "nfs" # Empty string must be explicitly set otherwise default StorageClass will be set / or custom storageClassName name
  volumeName: nfs-persistentvolume

Hope this helps

Bonus – run a Minecraft Bedrock inside K8S using your newly created PVC as storage

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mc-bedrock
  labels:
    app: mc-bedrock
spec:
  replicas: 1
  template:
    metadata:
      name: mc-bedrock
      labels:
        app: mc-bedrock
    spec:
      containers:
        - name: mc-bedrock
          image: itzg/minecraft-bedrock-server
          imagePullPolicy: Always
          resources:
            requests:
              cpu: 500m
              memory: 4Gi
          env:
            - name: EULA
              value: "TRUE"
            - name: GAMEMODE
              value: survival
            - name: DIFFICULTY
              value: normal
            - name: WHITE_LIST
              value: "false"
            - name: ONLINE_MODE
              value: "true"
            - name: ALLOW_CHEATS
              value: "true"
          volumeMounts:
            - mountPath: /data
              name: data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: nfs-persistentvolumeclaim
  selector:
    matchLabels:
      app: mc-bedrock
---
apiVersion: v1
kind: Service
metadata:
  name: mc-bedrock
  labels:
    app: mc-bedrock
spec:
  selector:
    app: mc-bedrock
  ports:
    - port: 19132
      protocol: UDP
  type: LoadBalancer

Get the IP assigned for the service

kubectl get service mc-bedrock -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Restart the pods in the deployment

kubectl logs -f deployment/mc-bedrock

#linux #k8s #kubernetes #pvc #pv #minecraft