I can understand why that would be your first thought, but isn't the actual lining fabric sewn into the parka inside-out, with the camo pattern on the reverse side? Also, the overall construction of the garment would make it very difficult to wear if one turned it inside-out.
I'd love to know the true story behind the N-3B Modified, but I keep thinking it must have to do with an excess of the camo fabric (or perhaps a shortage of the standard sage-green lining fabric). The only GI-issued item I've seen with that pattern is a helmet cover that was produced subsequent to the "Mitchell" pattern seen throughout the Vietnam period and thereafter. As I think about it, the early N-3B Mods didn't always have camo lining. Thus, it's possible that the Modified designation has more to do with the elimination of the nylon outer shell as the constant appears to be the use of nylon/cotton outer fabric.