VHDL Part 45 : Mask Generator 4

To which he patiently replied:

My "fast" answer without getting into details.
1) Have you tried this algoritme in other languages like C or Java.
2) Have you draw the data structures at a paper checking the flow. 
3) A got a feeling your about to create a huge combinatorial network which in the end will take up to much space.
May be should you consider a "pipelined" design in which you works at parts of the arrays. This will however cost you time instead of logic.

Jeppe