I think the drums are done that way quite intentionally. For a song like this to have a typical snare on the 2 and 4 would make it sound even more like Billie Jean Part II. What we hear instead is an interesting pattern where the kick hits on the 2 and the 4, while the shakers help the track move with a nice subtle shuffle. There's also some snare/clap hits going on in the chorus, a bit lower in the mix with some awesome delay/reverb. Great ambience in this track. It's got a bit of a reggae tinge to it, as I've said before, like Billie Jean meets Dirty Diana meets Spirits in the Material World.
It's the type of track that Michael does flawlessly, and should have done more often - simple, effective instrumentation that provides a vibe and an atmosphere while he does all the work in the vocal department. And man, do his vocals ever tear it up on this one.
Here's another track that I find to have a similar vibe going on in the rhythm section.