Orsdxy

Assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal", does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so I'm thinking it should do the same for covariance.

If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
$$
begin{aligned}
operatorname{Cov}(X + a, Y + b)
&= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
&= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
&= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
&= E[(X - E[X])(Y - E[Y])] \
&= operatorname{Cov}(X, Y).
end{aligned}
$$
Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.

Also, since correlation is defined as
$$
operatorname{Corr}(X, Y)
= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
$$
we can see that
$$
begin{aligned}
operatorname{Corr}(X + a, Y + b)
&= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
&= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
end{aligned}
$$
so in particular, correlation isn't affected by centering either.

That was the population version of the story. The sample version is the same: If we use
$$
widehat{operatorname{Cov}}(X, Y)
= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
$$
as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
$$
begin{aligned}
widehat{operatorname{Cov}}(X + a, Y + b)
&= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
&= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j - frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j - frac{n}{n} bright) \
&= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
&= widehat{operatorname{Cov}}(X, Y)
end{aligned}
$$
for any $a$ and $b$.

The definition of the covariance of $X$ and $Y$ is $E[(X-E[X])(Y-E[Y])]$. The expression $X-E[X]$ in that formula is the centered version of $X$. So we already center $X$ when we take the covariance, and centering is an idempotent operator; once a variable is centered, applying the centering process further times doesn't change it. If the formula didn't take the centered versions of the variables, then there would all sort of weird effects, such as the covariance between temperature and another variable being different depending on whether we measure temperature in Celsius or Kelvin.

"somewhere" tends to be a rather unreliable source...

Covariance/correlation are defined with explicit centering. If you don't center the data, then you are not computing covariance/correlation. (Precisely: Pearson correlation)

The main difference is whether you center based on a theoretical model (e.g., the expected value is supposed to be exactly 0) or based on the data (arithmetic mean). It is easy to see that the arithmetic mean will yield smaller Covariance than any different center.

However, smaller covariance does not imply smaller correlation, or the opposite. Assume that we have data X=(1,2) and Y=(2,1). It is easy to see that with arithmetic mean centering this will yield perfectly negative correlation, while if we know the generating process produces 0 on average, the data is actually positively correlated.
So in this example, we are centering - but with the theoretical expected value of 0.

This can arise easily. Consider we have a sensor array, 11x11, with the cells numbered -5 to +5. Rather than taking the arithmetic mean, it does make sense to use the "physical" mean of our sensor array here when looking for the correlation of sensor events (if we enumerated the cells 0 to 10, we'd use 5 as fixed mean, and we would get the exact same results, so that indexing choice disappears from the analysis - nice).