If you built your stadium in season 2 or later you get the "late start bonus". While you have this bonus, your daily popularity gain and the popularity increase from matches will be doubled. Popularity from selling tickets is not affected.
For each entire season you missed you get one season of late start bonus, up to a maximum of 3. If you start in season 2 you get the bonus for 1 season, if you start in season 3 you get it for 2 seasons, if you start in season 4 or later you'll get the bonus in your first 3 seasons.
This bonus depends on the moment you build your stadium since that's when you start gaining fans. If you build it a few days before the start of the season it will still count as being in the new season though.