Category Archives for commandline

Install Gearman from source on Ubuntu 18.04

I tried to install Gearman on my Ubuntu 18.04 Docker container and encountered a lot of issues.

This write-up is to save you some valuable time of your life.

TL;DR

apt update
apt install -y g++ uuid-dev gperf libboost-all-dev libevent-dev curl 

cd /tmp
curl -LO https://github.com/gearman/gearmand/releases/download/1.1.18/gearmand-1.1.18.tar.gz
tar xvzf gearmand-1.1.18.tar.gz
cd gearmand-1.1.18
./configure --with-boost-libdir=/usr/lib/x86_64-linux-gnu
make
make test
make install

# Start gearman in the background
gearmand --log-file /var/log/gearmand.log --daemon

To get there, I encountered the following issues:

configure: error: Unable to find libevent

./configure --with-boost-libdir=/usr/lib/x86_64-linux-gnu
...
checking test for a working libevent... no
configure: error: Unable to find libevent

Fix: apt install -y libevent-dev

configure: error: Could not find a version of the library!

./configure --with-boost-libdir=/usr/lib/x86_64-linux-gnu
...
checking whether the Boost::Program_Options library is available... yes
configure: error: Could not find a version of the library!

Fix: apt install -y libboost-all-dev

Installing the PHP client libraries for Gearman

If you intend to use PHP to talk to your Gearman service, use the following to get the libraries installed.

Note: you might not need the last line. This is to enable the Gearman PHP extension in Docker.

cd /tmp \
  && wget https://github.com/wcgallego/pecl-gearman/archive/gearman-2.0.6.tar.gz\
  && tar xvzf gearman-2.0.6.tar.gz \
  && mv pecl-gearman-gearman-2.0.6 pecl-gearman \
  && cd pecl-gearman \
  && phpize \
  && ./configure \
  && make -j$(nproc) \
  && make install \
  && cd / \
  && rm /tmp/gearman-2.0.6.tar.gz \
  && rm -r /tmp/pecl-gearman \
  && docker-php-ext-enable gearman

Automatisch bestanden verwerken en mailen met Hazel

Als ik een factuur binnenkrijg, wil ik dat deze netjes gearchiveerd wordt in de juiste directory én dat de factuur verstuurd wordt naar het speciale e-mailadres van mijn boekhouding zodat hij daar verwerkt kan worden.

Om dat handmatig te doen ben je niet lang bezig en het is ook niet moeilijk Maar het wordt wél vervelend om het steeds maar weer opnieuw te doen. Steeds weer dezelfde handelingen. Continue reading

Docker opschonen: vrije ruimte terugkrijgen

Als je Docker een tijdje gebruikt hebt, dan kan het zeker lonen om de boel eens op te schonen. Door items te verwijderen die je niet meer nodig hebt kun je een hoop ruimte vrijmaken.

TL;DR

Voer deze commando’s uit om alles op te ruimen

docker system prune
docker images --no-trunc | grep '<none>' \
    | awk '{ print $3 }' \
    | xargs docker rmi
docker ps --filter status=dead --filter status=exited -aq \
  | xargs docker rm -v
docker volume ls -qf dangling=true | xargs docker volume rm
Continue reading

Vagrant box updaten

Vagrant boxes zijn handige images van virtuele machines. Maar vaak zijn ze outdated en moet je ze eerst updaten voor je ze kunt gebruiken.

Ik doe daarom regelmatig een (automatische) update van mijn veelgebruikte images en bewaar ze centraal zodat ik snel een up-to-date image heb als ik wil gaan ontwikkelen.

TL;DR

Continue reading

Gitlab CI upload artifact fails: too large

Today I wanted to add a package-job to my Gitlab CI as instructed in this nice Gitlab tutorial.

I created the tar-file but when it came to uploading it failed with Request entity too large.

(...)
ERROR: Uploading artifacts to coordinator... too large archive  id=243 responseStatus=413 Request Entity Too Large status=413 Request Entity Too Large token=JYszbA9F
FATAL: Too large                                   
ERROR: Job failed: exit status 1

It took me some digging, but this is how I fixed this (note, the Nginx proxy was the one giving me a hard time).

Step 1: Set the maximum artifacts size

In your gitlab, go to Settings > Continuous Integration and Deployment > Maximum artifacts size (MB) and set it to the desired value. The default is 100MB.

Step 2: Set the nginx upload size

In the gitlab.rb file, mine at /etc/gitlab/gitlab.rb, set or uncomment the following line.

nginx['client_max_body_size'] = '250m'

 

 

And reconfigure gitlab to get this to work.

gitlab-ctl reconfigure

Step 3: (optional) update your proxy(!)

I run gitlab on docker containers. On the server, I run nginx as a proxy to redirect requests for gitlab to these containers.

I failed to update the proxy configuration to allow the POST-ing of data to the containers.

As I use nginx, this is the line I added. For Apache, just google and you’ll find your answer.

client_max_body_size 0;

This will set no limits on clients sending data.

For reference, this is my whole nginx vhost file.

server {
    listen 80;
    server_name git.pauledenburg.com;
    client_max_body_size 0;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Don’t forget to reload nginx.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

$ sudo service nginx reload

 

Change mysql_ to mysqli_ functions

In the process of upgrading PHP5.3 code I had to change all deprecated mysql_* functions to their mysqli_* counterparts.

For a lot of functions the signature stayed the same.

But mysqli_query and mysqli_connect have differences. So you can’t just find and replace them.

Instead of doing this manually, I wanted to find and replace recursively while changing the order of the arguments.

In vim:

# change mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
:%s/mysql_query(\(.\{-}\),\(.\{-}\))/mysqli_query(\2, \1)/g

Using sed:

# on linux

# mysql_query(param1, param2) to 
# mysqli_query(param2, param1)
sed -i 's|mysql_query(\(.*\),\(.*\))|mysqli_query(\2, \1)|g' devices.php

# on mac (otherwise you get the 'invalid command mode' when 
# you run the sed command)

# mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
sed -i '' -e 's|mysql_query(\([^,]*\),\([^)]*\))|mysqli_query(\2, \1)|g' devices.php

Recursively changing all files:

# in all files under current directory:
# mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
fgrep -rl mysql_query . | while read file; do
  sed -i '' -e 's|mysql_query(\([^,]*\),\([^)]*\))|mysqli_query(\2, \1)|g' $file
done

Note that sed cannot do non greedy matching.

That’s why we’re searching for anything but the separator until the separator like this:

# non greedy matching with sed
\([^,]*\),

It basically states: get everything except for the comma until you get a comma (which is the first one to appear).

1 2