Thankyou for your advice. You are right that R is my favourite language.
I have decided to use a matrix of real data decomposed into 'mutually orthogonal' components as the 'dependent' matrix, so that it contains real complexities such as structured non-uniform distributions.
and then I have produced synthetic independent matrices, which are each structured by different independent components of the dependent matrix.
So I know that multiple independent matrices should better constrain variance in the dependent matrix than any single independent matrix.
That sounds like a good idea! You could probably do clustering of the features (columns in your matrix I presume) into "dependence classes". Take one feature from each "dependence class" and you get a sort of an independent matrix. This could make it easy to construct different independent matrices to try out.
For the independent matrix, I would go through all triplets of features (columns), apply some function to the first two feature to construct a new feature, and then see how that new feature correlates with the third one. This is how I would go about trying to automatically find pairs of features that in combination correlate with the third one.
Though, idk if my ideas are useful or not as I'm not familiar with your exact problem, and there's no doubt a huge amount of domain-specific knowledge that you know but I don't that will heavily alter your approach to this. Regardless, it's a fun mental exercise to try to think about this, and anyways, it sounds like you are working on an interesting and non-trivial problem, so I wish you luck with this!