This fixes many problems we were having with a dynamically growing
array because of the realloc (pointers inside the area which was
allocated were no longer valid as soon as the realloc moved the
memory to another address).
Again, this is a rather big change, so expect problems and enable
core-dumps.