Stellar light curves are well known to encode physical stellar properties. Precise, automated and computationally inexpensive methods to derive physical parameters from light curves are needed to cope with the large influx of these data from space-based missions such as Kepler and TESS. Here we present a new methodology which we call The Swan, a fast, generalizable and effective approach for deriving stellar surface gravity (logg) for main sequence, subgiant and red giant stars from Kepler light curves using local linear regression on the full frequency content of Kepler long cadence power spectra. With this inexpensive data-driven approach, we recover logg to a precision of < 0.02 dex for 13,822 stars with seismic logg values between 0.2-4.4 dex, and < 0.11 dex for 4,646 stars with Gaia derived logg values between 2.3-4.6 dex. We further develop a signal-to-noise metric and find that granulation is difficult to detect in cool main sequence stars, in particular K dwarfs. By combining our logg measurements with Gaia radii, we derive empirical masses for 4,646 subgiant and main sequence stars with a median precision of 6.8%. Finally, we demonstrate that our method can be used to recover logg to a similar precision for TESS-baseline of 27 days. Our methodology can be readily applied to photometric time-series observations to infer stellar surface gravities to high precision across evolutionary states.