According to Jeremiah, God is speaking about the king and the people living in the city, as well as those who didn't go into captivity with you. He's letting everyone know what He thinks and what might happen to those people.
Jeremiah 29:16: [Know] that thus saith the LORD of the king that sitteth upon the throne of David, and of all the people that dwelleth in this city, [and] of your brethren that are not gone forth with you into captivity;
This verse contradicts Jeremiah 29:16 by promising that God will build up and not pull down the people, whereas Jeremiah 29:16 speaks of the people being delivered to ruin.
Jeremiah 24:6: For I will set mine eyes upon them for good, and I will bring them again to this land: and I will build them, and not pull [them] down; and I will plant them, and not pluck [them] up.