Rav Elazar Menachem Shach heard an idea from Isaac Zev Soloveitchik that I wanted to present here

The preliminaries are these: A mishna  says land conquered by Jews coming out of Egypt  but not settled by Jews returning to Israel from Babylonia is נאכל ואינו נעבד eaten but not worked on the seventh year. Another Mishna says עבר הירדן is obligated in ביעור. Fruits from lands beyond the Jordan river is not allowed to be eaten if there is nothing left of it in the fields. The Gemara in Yevamot 16b says אמון ומואב מעשרין מעשר עני בשביעית. Amon and Moav give tithes to the poor in the seventh year.
A few lines later the Talmud explains the reason: דאמר מר הרבה כרכים כבשו עולי מצרים והינחום עולי בבל שקדושה ראשונה קדשה לשעתה ולא קדשה לעתיד לבא כדי שיסמכו עליהם עניים בשביעית

Those are three facts from the Talmud.

The next three facts you need are these. Three Rambams. הלכות שביעית ויובל ד:כו. Land up until Kaziv is עולי בבל. After Kaziv is עולי מצרים. And is נאכל ואינו נעבד eaten but not worked on the seventh year.
The Rambam models his law here on the Mishna but adds ספיחים are eaten. [Not like ר' שמשון].
In the first chapter of Trumah the Rambam decides the law קדושה ראשונה the first sanctification did not sanctify the land permanently. Only the second sanctification did that.
In הלכות ביכורים ו:ה The Rambam says Amon and Moav and Egypt give tithes to the poor in the seventh year and Babylon gives the second tithe.

These are the six facts you need. Three from the Rambam and three from the Talmud.

Zev Soloveitchik told Rav Shach that land conquered by Jews coming from Egypt is obligated in all obligations that the land of Israel is obligated in.
One idea explains and clarifies everything in one simple sentence. I do have I think a very good question on this idea but I will save that for desert.

The shock value here lies starts when you notice the Gemara in Yevamot never said anything about land conquered by Jews returning from Egypt as being obligated in tithes to the poor. All it says is so that the poor will depend on them in the seventh year. That means it has all the obligations of the land of Israel. Seventh year, Truma, the forgotten sheaf etc., and etc.

The question I have is the fact that the Talmud says "they left them so the poor can depend on them in the seventh year." But all the more so if they had not left them then the land would have the holiness of the land of Israel and the poor would depend on them in the seventh year. My learning partner answered it is referring to ספיחים. But I think that is not a good answer because they left those lands before there was an decree against  ספיחים

 Clearly עולי מצרים is considered the land of Israel to the Rambam. And just like  Isaac Soloveitchik suggested it is obligated in all obligated of the land of Israel

A key fact here is הלכות תרומות א:כו that even the second sanctification did not sanctify any part of Israel until all Israel returns. Until then all obligations are by rabbinical decree.

 This explains the Rambam in laws of Trumot ch. 1 halacha 5, ולא פטרום כדי שיסמו כו עליהם עניים בשביעית. Logically that means if they had not left them poor people could not depend on them. At first glance this sounds senseless. But what I suggest it means is this : if they had not left them they would be obligated in the seventh year laws.

Still  what is hard to understand here is this way the Rambam puts it. He could have written that  עולי מצרים have an obligation of Trumah but not the seventh year and that is from the sages.