Babylon was not only a literal city and empire, but also a symbol of oppressive captivity. Thus it was not unusual for first century Jews to refer to Rome as Babylon. It is in this sense that we see Babylon referred to as emblematic of a great power oppressing true Christians in Revelation: “And upon her forehead was a name written, MYSTERY, BABYLON THE GREAT, THE MOTHER OF HARLOTS AND ABOMINATIONS OF THE EARTH” (Revelation 17:15).
Just as many literal Israelites fled literal Babylon when the decree of Cyrus (Ezra 1:1) granted them the freedom to do so, Jeremiah predicts that many Christians (symbolic Israelites) leave the oppressive system that persecuted them when the greater-than-Cyrus (the Messiah, Isaiah 45:1) gives them the same opportunity. Just as it was only a minority of Jews who left literal Babylon, so it only a minority of Christians will heed the call to flee mystic Babylon.