A Universe with finite age also has a finite causal scale. Larger scales cannot affect our local measurements or modeling, but far away locations could have different cosmological parameters. The size of our causal Universe dependson the details of inflation and is usually assumed to be larger than ourobservable Universe today. To account for causality, we propose a new boundary condition, that can be fulfill by fixing the cosmological constant (a free geometric parameter of gravity). This forces a cancellation of vacuum energy with the cosmological constant. As a consequence, the measured cosmic acceleration can not be explained by a simple cosmological constant or constant vacuum energy. We need some additional odd properties such as the existence ofevolving dark energy (DE) with energy-density fine tuned to be twice that ofdark matter today. We show here that we can instead explain cosmic accelerationwithout DE (or modified gravity) assuming that the causal scale is smaller thanthe observable Universe today. Such scale corresponds to half the sky at z=1 and 60 degrees at z=1100, which is consistent with the anomalous lack of correlations observed in the CMB. Late time cosmic acceleration could then be interpreted as the result of primordial Inflation.