SonarQube with Postgres on docker-compose

[updated 2022-08-08]

Struggling to get a working environment with SonarQube and PostgreSQL?

Use the following docker-compose file and be up and running in minutes.

It is as ‘bare’ as possible:

• use of official Docker images for both PostgreSQL and SonarQube
• no other configuration required
• use of volumes so you can backup your data

Recommended system specs

• >= 3GB of RAM

# file: docker-compose.yml

version: "3"

services:
  sonarqube:
    image: sonarqube:9-community
    # platform: linux/amd64  # uncomment this when using Mac M1
    restart: unless-stopped
    environment:
      - SONARQUBE_JDBC_USERNAME=sonar
      - SONARQUBE_JDBC_PASSWORD=v07IGCFCF83Z95NX
      - SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonarqube
    ports:
      - "9000:9000"
      - "9092:9092"
    volumes:
      - sonarqube_conf:/opt/sonarqube/conf
      - sonarqube_data:/opt/sonarqube/data
      - sonarqube_extensions:/opt/sonarqube/extensions
      - sonarqube_bundled-plugins:/opt/sonarqube/lib/bundled-plugins

  db:
    image: postgres:14.4
    # platform: linux/amd64  # uncomment this when using Mac M1
    restart: unless-stopped
    environment:
      - POSTGRES_USER=sonar
      - POSTGRES_PASSWORD=v07IGCFCF83Z95NX
      - POSTGRES_DB=sonarqube
    volumes:
      - sonarqube_db:/var/lib/postgresql
      # This needs explicit mapping due to https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
      - postgresql_data:/var/lib/postgresql/data

volumes:
  postgresql_data:
  sonarqube_bundled-plugins:
  sonarqube_conf:
  sonarqube_data:
  sonarqube_db:
  sonarqube_extensions:

Start this stack with the following command:

# start the containers
docker-compose up -d

You can reach your SonarQube instance at http://localhost:9000

Use the default credentials admin/admin to login.

Useful links:

• SonarQube on Docker Hub: https://hub.docker.com/_/sonarqube/
• PostgreSQL on Docker Hub: https://hub.docker.com/_/postgres/

Change mysql_ to mysqli_ functions

In the process of upgrading PHP5.3 code I had to change all deprecated mysql_* functions to their mysqli_* counterparts.

For a lot of functions the signature stayed the same.

But mysqli_query and mysqli_connect have differences. So you can’t just find and replace them.

Instead of doing this manually, I wanted to find and replace recursively while changing the order of the arguments.

In vim:

# change mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
:%s/mysql_query(\(.\{-}\),\(.\{-}\))/mysqli_query(\2, \1)/g

Using sed:

# on linux

# mysql_query(param1, param2) to 
# mysqli_query(param2, param1)
sed -i 's|mysql_query(\(.*\),\(.*\))|mysqli_query(\2, \1)|g' devices.php

# on mac (otherwise you get the 'invalid command mode' when 
# you run the sed command)

# mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
sed -i '' -e 's|mysql_query(\([^,]*\),\([^)]*\))|mysqli_query(\2, \1)|g' devices.php

Recursively changing all files:

# in all files under current directory:
# mysql_query(param1, param2) to: 
# mysqli_query(param2, param1)
fgrep -rl mysql_query . | while read file; do
  sed -i '' -e 's|mysql_query(\([^,]*\),\([^)]*\))|mysqli_query(\2, \1)|g' $file
done

Note that sed cannot do non greedy matching.

That’s why we’re searching for anything but the separator until the separator like this:

# non greedy matching with sed
\([^,]*\),

It basically states: get everything except for the comma until you get a comma (which is the first one to appear).

Disable xdebug for one run

This script disables xdebug for one run. No more error-messages like:

$ composer update
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug

and:

$ php-cs-fixer fix --dry-run .
You are running PHP CS Fixer with xdebug enabled. This has a major impact on runtime performance.
If you need help while solving warnings, ask at https://gitter.im/PHP-CS-Fixer, we will help you!

This is what you’ll get

We’ll create a script which will:

• disable xdebug
• run your command
• enable xdebug

the script we’ll name php-no-xdebug (or whatever you like)

With Xdebug (note the last line)

$ php --version
PHP 7.1.10 (cli) (built: Oct  6 2017 01:08:19) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans

Without Xdebug (note the missing last line)

$ php-no-xdebug --version
PHP 7.1.10 (cli) (built: Oct  6 2017 01:08:19) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies

The script php-no-xdebug

Create the script /usr/local/bin/php-no-xdebug with the following contents.

# file /usr/local/bin/php-no-xdebug
#!/bin/bash

php=$(which php)

# get the xdebug config
xdebugConfig=$(php -i | grep xdebug | while read line; do echo $line; exit; done)

# no xdebug? Nothing to do!
if [ "$xdebugConfig" == "" ]; then
    $php "$@"
    exit
fi

# get the configfile (which should be the first value)
# so strip off everything after the first space of the xdebug-config
xdebugConfigFile=$(php -i | grep xdebug | while read line; do echo $line; exit; done)

# test whether we got it right
if [ ! -f "$xdebugConfigFile" ]; then
    echo "No XDebug configfile found!"
    exit 1
fi

# disable xdebug by renaming the relevant .ini file
mv ${xdebugConfigFile}{,.temporarily-disabled}

# dissect the argument to extract the first one (which should be a script or an application in $PATH) from the rest
index=0
for arg in $(echo $@ | tr ' ' "\n")
do
    if [ "$index" == "0" ]; then
        firstArg=$arg
    else
      restArg="$restArg $arg"
    fi

   ((index++))
done

# check whether the command to be executed is a local PHP file or something in the $PATH like composer or php-cs-fixer
fullPath="$(which $firstArg)"
if [ "$fullPath" == "" ]; then
    # check whether it's a local file
    if [ ! -f  $firstArg ]; then
        echo "Could not find $firstArg. No such file or directory"
        exit 1
    else
        # just run the commands
        $php $@
    fi
else
    # run the command with the fullpath followed by the rest of the arguments provided
    $php $fullPath $restArg
fi

# execute the command
$php "$@"

# re-enable xdebug
mv ${xdebugConfigFile}{.temporarily-disabled,}

# test whether the conf file is restored correctly
if [ ! -f "$xdebugConfigFile" ]; then
    echo "Something went wrong with restoring the configfile for xdebug!"
    exit 1
fi

and make it executable

$ chmod +x /usr/local/bin/php-no-xdebug

That’s it! Run it like this:

$ php-no-xdebug composer update

Switch between multiple PHP versions on your mac

This article is a slimmed-down, firing-from-the-hips, get right to the action version based on the ones listed below. If you miss some background info or want a more spelled out version, please do visit these articles:

• https://getgrav.org/blog/macos-monterey-apache-multiple-php-versions

Install multiple PHP

If you want to install deprecated versions of PHP (< PHP7.2), then you’ll need to add this tap to homebrew:

brew tap shivammathur/php

#versions=(php@7.2 php@7.3 php@7.4) # in ZSH
versions="php@7.2 php@7.3 php@7.4 php@8.0 php@8.1" # in bash
for version in $versions; do
    echo "installing ${version}"
    brew install shivammathur/php/${version}
done

# install xdebug for php >= 7.2
pecl uninstall -r xdebug
pecl install xdebug

If you receive the error configure: error: Cannot find libz you need to install required libraries via XCode:

xcode-select --install 
brew upgrade

Install easy switch-script

To easily switch PHP versions, install the following script.

curl -L https://gist.githubusercontent.com/rhukster/f4c04f1bf59e0b74e335ee5d186a98e2/raw > /usr/local/bin/sphp
$ chmod +x /usr/local/bin/sphp

As I don’t add Apache on my host machine, change
apache_change=1 to apache_change=0 in the script.

vi `which sphp`

Switch PHP-version

Without arguments, this command returns useful information like which versions are installed and which is the active one right now:

$ sphp 7.4

If you need to enter your administrator password, then you probably need to disable the part where Apache is restarted (see chapter above).

Switching to php@7.4
Switching your shell
Unlinking /usr/local/Cellar/php@5.6/5.6.40... 0 symlinks removed
Unlinking /usr/local/Cellar/php@7.0/7.0.33... 0 symlinks removed
Unlinking /usr/local/Cellar/php@7.1/7.1.33... 0 symlinks removed
Unlinking /usr/local/Cellar/php@7.2/7.2.31_1... 0 symlinks removed
Unlinking /usr/local/Cellar/php@7.3/7.3.19... 25 symlinks removed
Unlinking /usr/local/Cellar/php/7.4.7... 0 symlinks removed
Linking /usr/local/Cellar/php/7.4.7... 24 symlinks created

PHP 7.4.7 (cli) (built: Jun 12 2020 00:04:10) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.7, Copyright (c), by Zend Technologies

All done!

NOTE FOR XDEBUG: if you want to use xdebug you’re switch command needs to be expanded to:

sphp 7.4 && pecl uninstall -r xdebug && pecl install xdebug

Now you’ll see that xdebug is enabled for PHP:

php -v
                  
PHP 7.4.7 (cli) (built: Jun 12 2020 00:04:10) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Xdebug v2.9.6, Copyright (c) 2002-2020, by Derick Rethans
    with Zend OPcache v7.4.7, Copyright (c), by Zend Technologies

That’s it!

Again; take a look at the great, kept up-to-date, article of Andy Miller over at his website: https://getgrav.org/blog/macos-monterey-apache-multiple-php-versions

Use events in symfony

The other day I had to add some logic right after an user was saved to the database. I ended up using events to get this done without cluttering the entity.

I first added the logic in the User-entity but I then realised this logic was not really related to the user entity itself. Or the application, for that matters.

In order to seperate concerns, I decided to create a hook after the User save-action. That would allow me to add logic at that particular time without cluttering the User entity with nonrelevant code.

This has a downside though. When you want to debug what the heck happens after the User is saved to the database, you won’t find it in the User entity. This might send you down a long code-hunt. But you will see the dispatching of the event though. So if you’re new to this, remind yourself that there can be a whole different world behind the dispatching of an event.

Setting it up consists of 3 steps:

1. create an Event class (the one that will be dispatched)
2. dispatch the event at the right time and place (after the User is saved to the database)
3. create (and subscribe) the subscriber which will take action upon the dispatched event

Create Event

The event is nothing more than a class.

The event is the object which is passed around. Therefore you want to populate the event with all the information the subscriber(s) need.

For that purpose I create a setter and a getter. The code which will dispatch the event will use the setter, the event subscriber will use the getter.

# file src/AppBundle/Event/UserCreatedEvent.php
<?php
namespace AppBundle\Event;

use AppBundle\Entity\User;
use Symfony\Component\EventDispatcher\Event;

class UserCreatedEvent extends Event
{
    private $user;

    public function setUser(User $user)
    {
        $this->user = $user;
    }

    public function getUser()
    {
        return $this->user;
    }
}

Dispatch Event

Now we decide in what moment of time we’ll dispatch (fire) the event. In our example this will be right after the user is saved to the database.

# file src/AppBundle/Entity/User.php
/* other code */

public function createUser(User $user)
{

    /* more code */

    // save the user to the database
    $this->em->persist($user);
    $this->em->flush();

    // dispatch an event where others can work with the newly created user
    $eventDispatcher = $this->container->get('event_dispatcher');
    $event = new UserCreatedEvent();
    $event->setUser($user);
    $eventDispatcher->dispatch('user.event.created', $event);

    return $user;
}

Act on event with the subscriber

Create subscriber

The subscriber is the class with the method which gets called once the event gets dispatched.

# file src/AppBundle/EventSubscriber/UserCreatedSubscriber.php
<?php
namespace AppBundle\EventSubscriber;

class UserCreatedSubscriber
{
    public function newUserCreated(UserCreatedEvent $event)
    {
        var_dump($event->getUser());
    }
}

Subscribe the subscriber

Now that we have the code for the subscriber, we need to actually subscribe the subcriber to the event. This is the glue between dispatching an event and acting upon it.

You do this in services.yml as you register it as a service.

# create a listener for the UserCreatedEvent
valuation.event.created:
  class: AppBundle\EventSubscriber\UserCreatedSubscriber
  tags:
    - { name: kernel.event_listener, event: valuation.event.created, method: newUserCreated }

That’s it!

Sources

If you want to read more:

• Symfony documentation:
  https://symfony.com/doc/current/event_dispatcher.html
• Nice dense setup: https://stackoverflow.com/a/34162603

behat: element is not clickable at point xxxx

When testing with behat on my laptop I often get the following error shoved up my face:

unknown error: Element <button id="mybutton">...</button> is not clickable at point (126, 698). Other element would receive the click: <span class="sf-toolbar-value sf-toolbar-info-piece-additional">...</span>

Cause: another element is in front of my button. My laptop has a smaller screen what makes this happen. In this case it is the Symfony toolbar which is in front of my button as I am testing it on my development environment.

Solution: scroll! (or maximize your browser window when it pops up)

# file FeatureContext.php

/**
 * Scroll HTML element  into view
 *
 * @Then I scroll element :cssSelector into view
 */
public function iScrollElementIntoView($cssSelector)
{
    // scroll the element
    $this->scrollHtmlElementIntoView($cssSelector);
}

/**
 * Scroll HTML element with the supplied ID in view so that you can click on it (for example)
 */
public function scrollHtmlElementIntoView($cssSelector)
{
    $function = <<<JS
(
    function(){
      let elem = $('$cssSelector');
      $('html, body').animate({scrollTop:elem.offset().top})
    }
)()
JS;
    try {
        $this->getSession()->executeScript($function);
    } catch (Exception $e) {
        throw new \Exception("ScrollIntoView failed");
    }
}

And you can use the following step in your test by using a CSS-selector :

# file: FeatureContext.php

Then I scroll element "button#coiCheckButton" into view

phpunit: method serves different output based argument

I had the issue that my test-double sent incorrect values when invoked with specific arguments.

It returned null on every request.

First I thought this mock would return null on every call, but that’s not the case. Then I stumbled on this post on StackOverflow https://stackoverflow.com/questions/12748607/phpunits-returnvaluemap-not-yielding-expected-results.

The solution: the map-array needs all parameter-values listed in every element. Even the optional ones. I had to add null values for the optional parameters!

use returnValueMap method to map the received arguments to an array.

function myMethod($name, $optional=null){
  // ...
}

// Define which value need to be returned when 
// called with argument 'x'
// first element is argument 'x'; the argument passed, 
// 2nd element is the optional argument of the function
// 3d element is what will be returned by the test-double
$map = [
  ['value1', null, $valueToReturn1],
  ['value2', null, $valueToReturn2]
];

$request->expects($this->any())
  ->method('getRepository')
  ->will($this->returnValueMap($map));

$request->getRepository('value1'); // will return $valueToReturn1

Without the null added to it, it won’t work.

Example for my mock with Symfony. This will return the relevant repository on a request for method getRepository and with returnvalues for arguments AppBundle:Valuation and AppBundle:ObjectInvolvement.

$map = [
    ['AppBundle:Valuation', $valuationRepository],
    ['AppBundle:ObjectInvolvement', $objectInvolvementRepository]
];
$entityManager->expects($this->any())
    ->method('getRepository')
    ->will($this->returnValueMap($map));

 

Too large textareas

The other day I noticed really large textareas on the website. The textarea contained all the text and a lot of whitespace after it. After a short investigation I noticed two libraries fiddling with the height of textareas on the website: autosize and autogrow (a CKEditor plugin).

As the names suggests, they grow your textareas to the size of the text so you don’t have to scroll inside the textarea itself.

Cause

The large textareas reside on tabs which are not visible on pageload. And it were these textareas which grew out of bounds.

The cause is the following:

When an HTML-element is hidden, it is not possible for javascript to accurately get it’s width. And the width is needed to calculate correct dimensions for the textarea.

As noted on the autosize website (at the bottom) there are few things you can do:

• assign a set width in pixels to the textarea in your stylesheet

• delay autosizing until the element is revealed:

  var ta = document.querySelector(‘textarea’); ta.addEventListener(‘focus’, function(){ autosize(ta); });

• use the autosize.update method after the element is revealed

The textareas I was dealing with did not have the issue with autosize, but rather with the autogrow plugin from CKEditor. The cause is the same.

Solution

My solution was to initiate the CKeditor on all visible textareas on the page. By putting this initiation in a function, I could call this function again when the contents of a new (bootstrap3) tab were loaded. But then again, a tab itself can contain hidden elements as well. So as a third I used library VisSense to observe relevant textareas when they become (partly) visible. I added a callback on this function to initiate CKEditor whenever the field becomes visible.

<!-- include the VisSense library -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/vissense/0.10.0/vissense.min.js"></script>

// get all relevant textareas (eg: textareas which have the 
//attribute 'data-provide' with value 'wysiwyg')
let ckeditorElements = $('textarea[data-provide="wysiwyg"]');

// add a listener which instructs to add CKEditor when the 
// field comes into view
$(ckeditorElements).each(function (index, element) {
    VisSense.VisMon.Builder(VisSense(element))
        .on('visible', function (monitor) {
            addCKEditorToElement(element);
        })
        .build()
        .start();
});

 /**
 * Add CKEditor functionality to an HTML-element 
 * (usually a textarea)
 * @param element   HTML-element like $('div.some-class')
 */
function addCKEditorToElement(element)
{
   let ckeditorTemplate = wysiwygConfigSmall;

   CKEDITOR.config.extraPlugins = 'autogrow';

    // instruct CKEditor to takeover this textarea
    CKEDITOR.replace(element);
}

Symfony JsTranslationBundle shows javascript translations upon login

In order to benefit from the Symfony3 translations in javascript files, I use BazingaJsTranslationBundle from

I had the weirdest thing: whenever I logged out, cleared my cache, went to the login page and logged in, I got redirected to /translations which showed me the generated JSON file with the translations.

Cause: the login page tried to load the JSON translation because of my <script src="{{ url('bazinga_jstranslation_js', {}) }}?locales=en,nl"></script> script import. But that failed as security did not allow anonymous access to that URI.

Symfony registered the failed URL and, as I’m on the login page, redirected me to the url that first failed once I was logged in.

Solution: allow anonymous access to URI /translations.

In file app/config/security.yml I added the following:

access_control:
- { path: ^/login, roles: IS_AUTHENTICATED_ANONYMOUSLY }
- { path: ^/translations, roles: IS_AUTHENTICATED_ANONYMOUSLY }

Ansible stat module

I use the Ansible stat module mostly for conditional statements.

For example, only when directory /var/www/phpmyadmin does not exist include the task for installing PHPMyAdmin.

Only when <directory> exists; <do this>

But of course, there are a lot of other uses for this module then the one I use most.

It does this by doing the following two things:

1. get all system-information about the directory
2. execute statement, depending on values from the fetched system-information

But there are a lot of uses for the stat module.

The official documentation for the stat module is on, ofcourse, the Ansible website: http://docs.ansible.com/ansible/stat_module.html

full example

# file: install.yml

# install PHPMyAdmin when it's not installed already
- name: PHPMyAdmin | check the need for installment
  stat:
    path: /var/www/html/phpmyadmin
  register: pma

# Only include the PHPMyAdmin installation-task when it's not installed yet
# e.g. the directory /var/www/html/phpmyadmin does not exist
- include: PHPMyAdmin | install
  when: pma.stat.isdir is not defined

As you can see, we’re checking against the variable pma which holds the stats information for our path /var/www/html/phpmyadmin. This variable holds a lot more then just the isdir property.

The full list of information returned by the Ansible stat module:

• atime
• attributes
• charset
• checksum
• ctime
• dev
• executable
• exists
• gid
• gr_name
• inode
• isblk
• ischr
• isdir
• isfifo
• isgid
• islnk
• isreg
• issock
• isuid
• lnk_source
• md5
• mime_type
• mode
• mtime
• nlink
• path
• pw_name
• readable
• rgrp
• roth
• rusr
• size
• uid
• wgrp
• woth
• writeable
• wusr
• xgrp
• xoth
• xusr

I will continue to update this page as needed.