Ultra-short period planets provide a look at the inner edge of the allowed parameter space for planetary orbits. One particularly intriguing geometry of system containing ultra-short period planets is high multiplicity systems where the ultra-short period planet and the outer planets exist in two different dynamical states. This has manifested in the observational data as a small number of stars hosting systems of tightly packed coplanar inner planets as well as an ultra-short period planet, where the orbit of the latter is misaligned relative to the mutual plane of the former. We describe two different mechanisms that can produce an ultra-short period planet that is misaligned with the rest of its compact planetary system: natural decoupling between the inner and outer system via the stellar quadrupole moment, and decoupling forced by an external companion with fine-tuned orbital parameters. These two processes operate at different timescales, and can thus occur simultaneously or independently within a single system. We use the K2-266 system as an example to illustrate the dynamics of these two processes. We will also discuss the possibility of placing constraints on when ultra-short period planets in multi-planet systems arrive at their final orbital locations using the results of this work.