My team is familiar with a bunch of plays that I don't run. Is this because I check them off when I'm scouting my opponents?

It depends - your offense gains familiarity with the defensive plays you scout and vice versa, but your offense learns plays based on what you call yourself. It's possible that your players have learned the plays elsewhere, and during training camp your team works on all the plays in your playbook. If you are in a young league, veteran players are also assigned a random value for their various play knowledge.

I have the same situation.. had wondered.

also have a related question but I will start it in a new thread.