The COVID-19 pandemic has had an impact on the routines of children and adolescents and on their level of involvement in physical activity (PA). The restrictive rules applied in this period affected the functioning of physical education (PE) classes in Portugal, and strongly limited student participation. The aim of this study was to analyze and compare the affective attitude (enjoyment) of adolescents during face-to-face PE lessons during the COVID-19 pandemic (from September 2020), according to sex, education, and PA levels. The study included 1369 students (621 boys and 748 girls), aged 12–18 years, Mage: 14.4; SD: 1.74. A validated online questionnaire was distributed between November and December 2020, and the data were analyzed for positive and negative affective attitude, using MANCOVAs (multivariate analysis of covariance) adjusted for sociodemographic and behavioral variables. The results showed higher negative affective attitudes in younger boys when compared to older boys and to girls in the same education level. Younger less active boys also showed higher negative affective attitudes than less active girls.