Developing a Media Literacy Perception Scale

The purpose of this research is to develop a scale to determine the perception of media literacy. For this purpose, trial forms were created and submitted for expert opinion. The scale items were revised based on the feedback received after trial forms were examined concerning the content validity according to expert opinion. A pilot study was conducted with a sample group, with characteristics similar to the general population, in order to assess the construct validity, sub-dimensions, and reliability level of the draft scale, which was finalized after expert opinions were obtained and appropriate revisions were made. The working group of the pilot study was composed of students selected apart from the research sample. In order to reach the students, the simple random sampling method was used. The construct validity of the scale was tested by performing an exploratory factor analysis and a confirmatory factor analysis (EFA + CFA). To determine the reliability of the scale, item-total score correlations and Cronbach’s alpha internal consistency coefficient values were used. Since several items had low item-total test correlations, and a number of items had loadings for two distinct variables that were close to one another, these items were gradually excluded from the study until there were no extreme values present in the analysis, and statistical errors were eliminated. Validity analyses (EFA + CFA) and reliability (Cronbach’s alpha) of the obtained data were performed, and the statements were finalized with the ‘Media Literacy Perception Scale’ (MLPS), consisting of sixteen items and three dimensions (factors), being created. The Cronbach’s alpha coefficient for the entire scale in this research was found to be .86.


INTRODUCTION
As media outlets have become more varied, people's access to, processing of, and transmission of information at the local, national, and international levels has changed over time.At the same time, media sectors and organizations have started to occupy more and more of our time at work and at home, thereby influencing public perception (Considine, 2009).The media plays an undeniably significant role in today's world.We face the risk of underestimating the social importance of mass media because we live in a society that is heavily reliant on it (Devereux, 2013).Media culture also controls modern Western civilization (Featherstone, 1995;Kellner, 2003).There is no reason to undervalue the power of the media.The ability to control the masses is inherent in excessive media, as the word suggests.As a result, the concept of media literacy is emerging in light of how important the media is to modern societies.
It seems that the media is trying to influence people in an insidious way.Most people are not aware of the methods being used to indoctrinate and position them by the media, because media teachings tend to be invisible and acquired unconsciously (Kellner & Share, 2005).We are exposed to media bombardment daily, from digital games to social networking sites, and this requires us to start assessing the impact of the media in shaping perceptions, beliefs, and attitudes (Thoman & Jolls, 2008).Children and young people are greatly influenced and directed by the media.It is therefore important to start teaching children and teenagers early on about the consequences of the media.
The field of media literacy is a relatively new research area.The emergence of new media services and platforms has significantly altered how society engages with the media.As this relationship has changed, the meaning of being literate has also changed.Traditional literacy learning involves developing the skills and knowledge of reading and interpreting texts, as well as writing at a competent level.Literacy is a skill that goes beyond the process of information and meaning-making (Vincent, 1999), and is no longer limited only to written or oral applications (Brown, 1998).The definition of literacy should be broadened to incorporate new abilities and represent the merger of old and new literacy (Verezub et al., 2008).Kellner and Share (2005) define literacy as the ability to read, interpret and create certain types of texts and works, as well as the acquisition of intellectual tools and capacities in one's full participation in culture and society.If we revisit the idea of literacy, we can say that media literacy is a type of multi-literacy that helps us understand how the media creates the messages that influence our culture (Kubey, 2003).
Young people and children are impacted in a variety of ways by excessive media exposure and consumption.Teenagers and children who are exposed to excessive media may experience attention issues, such as drug and alcohol abuse, weight problems like obesity, academic failure, smoking, and increased early-life sexual activity (Nunez-Smith et al., 2008).The media affects children in the negative ways mentioned above, because they do not have many world experiences to compare to what they see and hear (Strasburger, 2004).Therefore, their limited real-life experiences can impair their capacity to make wise decisions.Controlling how the media shapes our thoughts is a key aspect of media literacy.Media literacy helps people distinguish and evaluate media content, critically analyze media forms, research media influences and uses, use media wisely, and develop alternative media (Kellner & Share, 2005).The overarching goal of media literacy is to help young people understand the intention behind media messages and to learn how to evaluate the media with a critical eye (Chen, 2008).Media literacy education can facilitate pupils' reflections on their media experiences and help them to develop critical thinking skills.The necessity of media literacy in today's digital world is indisputably obvious.

The Purpose and Importance of the Research
This research aims to develop a scale to determine the perceptions of media literacy.Perception of media literacy is awareness of media literacy.A media literate person can actively use media tools and critically examine the media messages they are exposed to through the filter of the mind.Media literacy can be empowering learning for students, allowing them to become responsible citizens, while protecting them from possible media manipulation.Media literate individuals do not readily accept messages which are imposed on them by the media.
The achievements of the media literacy textbook, which is taught in secondary schools in Turkey, and similar scales in the field, were examined.However, any scale that fully covers all dimensions of media literacy has not been found.As a result, the need to develop a new media literacy scale has arisen.For this reason, it is thought that the developed scale would play an important role in measuring students' perception of media literacy.It is also thought that the study will contribute to media literacy education by bringing a new scale to the field.

METHOD Research Design
This research is a scale development study.In this section, research sample, data processing steps in the collection process, and the scoring of the media literacy perception scale are listed.

Research Sample
The study group of the research consisted of a total of 602 pre-service teachers, of which 328 were female and 274 were male, enrolled in the Social Studies teaching programs of the Faculty of Education at a state university in the Central Anatolia Region of Turkey.The ages of the participants ranged from 17 to 27.

Data Collection
The necessary permission was obtained for the implementation of the research, in order not to disrupt the education of the students.The data were collected from the students through the face-to-face interview method using paper-pencil.

The Process of Developing the Media Literacy Perception Scale Step 1: Creating an Item Pool
The scale is a closed-ended Likert scale.The national and international literature on media literacy, as well as the objectives of the media literacy textbook used in secondary schools run by the Ministry of National Education, were all reviewed during the development of the media literacy perception scale.
Without making any distinctions based on size, a 46-item item pool with positive and negative statements that are considered to be connected to the perception of media literacy was constructed in the first stage.In the second stage, three dimensions that are thought to define the concept of media literacy theoretically were created.While creating the first dimension, 'The Effect of Media on Society (Effect)', the relationship between media and society was investigated and the effect of the media on society, which is one of the fundamental elements of media literacy education and which is inevitable for almost all societies today, is set out. 'Metacognitive Media Awareness (Awareness)' in the second dimension is one of the basic components of media literacy in the literature.This dimension measures how well people perceive, evaluate, and analyze media communication, as well as how well they spot flaws and potentially harmful content.The third and final dimension is the 'Use of Media Tools (Use)' dimension.This dimension was developed with the intention of measuring which media tools, which is one of the key elements of media literacy, and for what purpose, how frequently, and how people use them.The expressions in the item pool and the dimensions were matched in the third stage.The final step involved creating a trial form using the dimensional expressions.
Step 2: Seeking Expert opinion (Content Validity) For the purpose of analyzing the statements in the trial form, eight experts were consulted.The experts included those for assessment and evaluation, psychological counseling and guidance, Turkish grammar, and media literacy course experts.The purpose of examining the scale by experts is to ensure the content validity of the scale (Büyüköztürk, 2018).Content validity is the extent to which a test covers the behavior that it is intended to be measured by the test (Baykul, 2015;Crocker & Algina, 1986).Content validity is related to the adequacy of the item sample; that is, it is related to the extent to which the expressions that are thought to reflect the psychological state to be measured reflect the content area (DeVellis, 2016).According to Tekin (1996) and Turgut and Baykul (2012), it can be said that a measurement tool has content validity, which adequately and balancedly exemplifies the scope of measurement, and each of the items it covers truly measures the behavior it wants to measure.
After the trial form was examined in terms of content validity, based on the expert opinion in this direction, the scale items were revised with the feedback received.The trial form was also checked to see whether the expressions contained in it had any deficiencies in terms of Turkish grammar rules.After the necessary corrections were made, the number of items, forty-six in the trial form, was reduced to twenty-eight, and the scale was rearranged and made ready for the pilot test.
Step 3: Pre-Testing (Pilot Test) A pilot study was conducted using a sample group, which had characteristics similar to the general population, in order to assess the construct validity, sub-dimensions, and reliability level of the draft scale, which was finalized after expert opinions were obtained and appropriate revisions were made.The working group of the pilot study was composed of students selected apart from the research sample.
The sample size needs to be several times larger than the number of items in order to collect valid and meaningful data on the items (Balcı, 2010).Nunnaly (1978) asserts that a sample group of 300 people is sufficient for a pilot study.Norusis (2005) also states that the sample size should be at least 300 for factor analysis.Likewise, Tabachnick & Fidell (2007) emphasize that a sample size of at least 250-300 people is required for factor analysis.Therefore, a pilot study was carried out on two separate groups of 326 (EFA) + 276 (CFA) people.In order to reach the students, the 'Simple Random Sampling' method was used.The construct validity of the scale was tested by performing a factor analysis (EFA + CFA).To determine the reliability of the scale, item-total score correlations and Cronbach's alpha internal consistency coefficient values were used.
Factor analysis is a statistical technique used to group variables that are thought to explain a formation or cause, and to group variables that affect this formation in order to reveal common factors by transforming related data structures into a smaller number of independent and new data structures (Özdamar, 2002).Büyüköztürk (2002) defines factor analysis as multivariate statistics aiming at revealing and discovering conceptually meaningful fewer new variables (factors or dimensions) by bringing together a large number of interrelated variables.Factor analysis consists of two main methods; exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).As a result, EFA is a method of examining if the data gathered under a certain factor actually represents the theoretical framework (Green et al., 2000).When performing EFA, the variables that are connected to one another are grouped to summarize and characterize the data; these variables may or may not be combined meaningfully.However, CFA is an extremely advanced technique used further in the research to test the theory regarding latent variables (Tabachnick & Fidell, 2007).CFA is used to evaluate construct validity (Floyd & Widaman, 1995;Kline, 2015).CFA is a method used to test hypotheses based on factor analysis.DFA is a type of analysis in which a predefined and constrained structure is tested to find whether it is validated as a model (Çokluk et al., 2014).Based on the data obtained from the measurement tool developed in accordance with a theoretical structure with CFA, an attempt is made to test whether the structure in question is verified and, therefore, this highly efficient method is frequently used to assess construct validity, as well as to generate new theories and evaluate the applicability of preexisting ones (Erkuş, 2003).The scale development process requires the use of EFA and CFA.First, the EFA approach should be used in the scale development process, followed by the CFA technique on a different data set; because CFA aims to 'test' this structure whereas EFA aims to 'understand' the psychological structure as its name implies (Costello & Osborne, 2005).CFA is used to determine whether the variable groups contributing to the 'k' number of factors determined in testing which factor is highly correlated with the variable groups obtained by EFA are adequately represented by these factors (Özdamar, 2002).CFA is described as a logical progression of the EFA model and is used for the psychometric evaluation of multi-item measuring tools while the scale is being developed (Ergin, 2010).

FINDINGS Item Analysis and Suitability of Data Set
In order to examine the multivariate normality of the data set, skewness and kurtosis values were checked before the factor analyses were performed to determine the construct validity of the MLPS.The current literature indicates that a value of between ∓1.5 (Tabachnick & Fidell, 2007) and ∓2.0 (George & Mallery, 2010) in the ratio of skewness and kurtosis values to their standard deviations is shown as evidence that the data set is normally distributed.It was seen that the skewness and kurtosis values of each dimension were between +1.4 and -1.2 values in the current study.Item-total correlations were also checked.It is important in order to reveal whether the items included in the draft scale are really suitable for the intended psychological structure (Erol & Ordu, 2018 ).It is agreed that items having an item-total correlation of .30and higher can be included in the scale when item-total correlation is employed in scale development studies (Büyüköztürk, 2018;Şencan, 2005;).The total scores of the scale and dimensions were checked along with the scores obtained from each study item, and it was discovered that all items had values above .30while the item total correlations were being evaluated.After this, in order to ascertain whether both data sets (EFA and CFA) were appropriate for factorization, the Kaiser Mayer Olkin (KMO) and Barlett Sphericity tests were conducted.The KMO must exceed .60 and the Barlett test must be significant in order to evaluate whether the data set is appropriate for factor analysis (Büyüköztürk, 2018;Tavşancıl, 2005).KMO values were found to be between 77 and 90, and Bartlett tests were found to be significant in the study.The result of the analysis reveal that the data sets were suitable for factorization.

Exploratory Factor Analysis (EFA)
In order to determine the construct validity of the draft scale, an EFA study was conducted first.The Maximum Likelihood Method (ML) was determined as the factor extraction method for EFA in this study, which is one of the factorization techniques giving the best fit for normally distributed data (Field, 2005).Since the factors have approximate meanings in social science research, and the factor structures are partially related, the Direct Oblimin technique, one of the oblique rotation methods, was used as the Rotation Method (Şencan, 2005).The current study employed the Eigenvalue method as the strategy for determining the number of factors (Büyüköztürk, 2018).While performing EFA, such criteria as the items (statements) to be included in each factor should be compatible with the theoretical background, the factor of the Eigenvalues should be 1 or more than 1, an expression should have a factor load of .30or more in the factor in which it is included, the difference between the load values in the factors in which the items are included and the load values in the other factors should be at least .10 or more (Büyüköztürk, 2018) were taken into consideration.The variance ratios explained by the factors with eigenvalues greater than 1, in line with the Kaiser-Guttman principle, were then investigated using the Total Variance Explained chart as a consequence of the loop less application of this technique (Zwick & Velicer, 1986).Table 1 below presents the results of the first factor analysis of the scale.Table 1 shows that three factors with eigenvalues greater than 1 were obtained as a result of the Exploratory Factor Analysis (Phase I) procedure.Considering the factors with eigenvalues of 1 and greater than 1, a 3-factor structure of the MLPS emerged (eigenvalues, respectively: 7.011, 2.007, 1.880).These three variables were found to account for 35.35% of the overall variation.However, the overall variance described by multifactorial scale structures is expected to range between 40% and 60%, according to related literature (Scherer et al., 1988;cited in Tavşancıl, 2005).Therefore, it seems that the first factor analysis of the draft scale does not adequately explain the structure that is being measured based on this criterion.Additionally, as many items had low item-total test correlations, and some items had loads for two distinct variables that were close to one another, these items were gradually removed from the analysis until no extreme values remained in the analysis.The following Table 2 provides the item and test statistics of the scale, which was created after the statistically inaccurate items were eliminated.As can be seen in Table 2, the scale items were categorized into three sub-factors as a consequence of the exploratory factor analysis (Phase II) procedure, which was statistically significant and which explained 49.86% of the total variance.The eigenvalue of the 'Effect of the Media on Society-(Effect)' (F1) factor consisting of six items (Q24, Q21, Q14, Q28, Q8, Q26) was found to be 5.25.This sub-factor alone explains 12.67% of the total score variance.The items in this factor had factor load values ranging from 0.577 to 0.747.The eigenvalue of the second factor, 'Metacognitive Media Awareness'-(Awareness) (F2) consisting of seven items (Q4, Q6, Q1, Q2, Q5, Q9, and Q12), was found to be 2.15.This sub-factor alone explains 19.08% of the total score variance of the relevant scale.The items in this dimension had factor load values ranging from 0.409 to 0.610.The eigenvalue of the third factor, 'Use of Media Tools'-(Use) (F3) consisting of four items (Q4, Q6, Q1, Q2, Q5, Q9, and Q12), was found to be 2.15.This factor alone explains 6.29% of the total score variance of the relevant scale.The items in this factor had factor load values ranging from 0.391 to 0.685.The item and test statistics observed after eleven items were removed from the draft scale were considered evidence for the structural validity of the and the scale structure of the three-factor MLPS, consisting of the remaining seventeen items, was theoretically found sufficient for measuring students' perceptions of media literacy.

Confirmatory Factor Analysis (CFA)
In order to determine whether the theoretical structure and the empirical structure are consistent, the construct validity of the MLPS was retested after the factorial structure was produced with the EFA using first level and second level CFA.The CFA evaluates the power of explaining the latent variables that it is assumed to measure using the manifest variables that make up a scale (Erol & Senturk, 2018).The CFA also enables the building of alternative models in line with the purpose of the study and allows for goodness-of-fit comparisons across these models (Drasgow & Schmitt, 2002).In other words, the CFA is used to check whether the relations hypothesized in the theoretical universe correspond to the data gathered by empirical observations.As a result, a model that simulates the three-factor structure produced by the EFA was designed and tested.To assess how well a model fits the data when it is tested with the CFA, there are a variety of fit indices available.It is advisable to combine numerous fit index values, since different fit indices have advantages and disadvantages when compared to one another in assessing the fit between the theoretical model and the actual data (Çokluk et al., 2014).
In this research, CFA analyses were conducted using standardized regression coefficients, chi-square/sd value, GFI, CFI, NFI, and RMSEA model fit indices.Standardized regression coefficients indicate the power of manifest variables to predict latent variables, that is, factor loadings.For standardized regression coefficients, .71and above are excellent, .63 is very good, .55 is good, .45 is acceptable, and .32 is poor (Tabachnick & Fidell, 2007).GFI, NFI, and CFI values greater than .90and RMSEA values less than .08show acceptable fit as a general rule.An acceptable number in terms of fit is the χ2/sd ratio, which is calculated by dividing the χ2value by the degrees of freedom between 2 and 3 (McDonald & Moon-Ho, 2002;Schermelleh-Engel, Moosbrugger & Müller, 2003;Cook & Thompson, 2000, cited in Şimşek, 2007).The following Figures 1, 2, and 3 show the CFA models that were created, while Table 3 below shows the goodness of fit values.

MLPS-Level 1 of CFA (Phase I)
As shown in Figure 1, the factor loads of the items in the scale dimensions range from 42 to 73, which includes the factor loads of the Amos model obtained from the first level CFA (phase I) for the MLPS.The fit index values obtained were found to χ2/sd=2,88; GFI= .90,NFI=.88,CFI=.92,RMSEA=.056(Table 3).Although the fit indices were partially sufficient, the factor load of an item (Q5) was found to be extremely low and the analysis was repeated by removing it from the model.
As can be seen in Figure 2, improvements were observed in the factor loads and fit index values in the Amos according to the first-level CFA (Phase II), which was performed again after the removal of item Q5.The factor loads of the items in the scale dimensions ranged from .53 to .75, according to the second analysis.The fit index values obtained were found to be χ2/sd=2,25; GFI= .95,NFI=.92,CFI=.95,RMSEA=.046(Table 3).In addition, changes were made between the 'Q28-Q26' and 'Q4-Q12' items.These findings suggest that the model fits with the data obtained from the sample, based on the fit indices obtained from the first level CFA (Phase II).

Figure 2. MLPS-Level 1 of CFA (Phase II)
As a result of the first level CFA, the factors of 'The Effect of Media on Society'-(Effect) (F1), 'Metacognitive Media Awareness'-(Awareness) (F2), and 'Use of Media Tools'-(Use) (F3) were independent of each other, but were components of the model.However, it is unknown whether the three factors are truly related to media literacy perception.However, Yurdugül & Aşkar (2008) state that certain of these factors may not be related to the designed model, while scale items can generate certain factors with the first-level CFA.Therefore, the secondlevel CFA was applied to the designed model to determine how well the MLPS sub-dimensions, whose factorial structure was confirmed by the first-level CFA, fit the implicit variable of 'media literacy perception', which was built as a higher structure, and to reveal its factorial validity.Figure 3 below depicts the Amos model factor loads obtained from the second level CFA.

Figure 3. The MLPS-Level 2 of the CFA
As can be seen in Figure 3, the factor loads of the items in the MLPS dimensions range from .53 to .75.Goodness of fit values were found to be χ2/sd=2,25; GFI=.95,NFI=.92,CFI=.95, (Table 3).As a result, the error variance, factor loadings, and predictive values calculated in the first and second level CFAs were found to be equal.This means that the second-order relations have no effect on the model's parameter values and fit indices, while the factors adapt to the 'Media Literacy Perception' superstructure.As can be seen in Figure 3, the most important component of media literacy perception was the 'Use of Media Tools'-(Use) (F3) factor (β=1.03; p<0.05), depending on the study sample.Respectively, this dimension is followed by the factors of 'Metacognitive Media Awareness'-(Awareness) (F2) (β=0.74;p<0.05) and 'The Effect of Media on Society'-(Effect) (F1) (β=0, 58; p<0.05).When these findings are taken into consideration collectively, it points out that the construct validity of the MLPS has been achieved.Level 2 of CFA Step 1: Reliability Reliability is a concept that reveals the degree of freedom from random errors in measurement results (Turgut & Baykul, 2012).Reliability is the degree to which a test or scale coherently and consistently measures what it intends to measure (Büyüköztürk, 2018).Cronbach's alpha (α) reliability coefficient, one of the most commonly used techniques for calculating scale reliability, is a single-application reliability calculation technique that can be used in cases where scoring with the scoring or the grading method is used (Tezbaşaran, 2008).If the Cronbach's alpha (α) reliability coefficient is between 0.00 and 0.40, the scale is not reliable; if it is between 0.40 and 0.60, the scale has low reliability; if it is between 0.60 and 0.80, the scale is reliable; and if it is between 0.80 and 1.00, the scale is highly reliable (Tabachnick & Fidell, 2007).In this study, the Cronbach alpha coefficient for the entire scale was .86; the dimension of media's effect on society was .82; the dimension of metacognitive media awareness was .74;and the dimension of media tool use was .73.The findings of the current study may conclude that the internal consistency of the general structure and sub-dimensions of the MLPS are sufficient and therefore reliable.
Step 2: Evaluation of the Scale The obtained data were subjected to validity (EFA + CFA) and reliability (Cronbach's alpha) analyses and the statements were finalized, and the 'Media Literacy Perception Scale'(MLPS) was thereby created, consisting of sixteen items and three dimensions (factors).
All items in the MLPS are scored as '1-Strongly Disagree', '2-Disagree', '3-Undecided', '4-Agree', and '5-Strongly Agree'.The scale expressions are used to assess students' perception of the exemplified situation, or the degree of agreement.A higher total score from any dimension shows that the students agree more with the statements in that dimension; a higher total score from the overall scale shows that the students' media literacy perceptions increase.
The first sub-dimension of the MLPS was named 'Media's Effect on Society'-Effect), which was measured by items Q24, Q21, Q14, Q28, Q8, and Q26.The items Q14 and Q26 in this dimension were reverse coded.The highest possible score from this dimension is 30, and the lowest possible score is 6.The second sub-dimension was named 'Metacognitive Media Awareness'-(Awareness), which was measured by items Q4, Q6, Q2, Q1, Q9, and Q12.The items Q6 and Q9 in this dimension were reverse coded.The highest possible score from this dimension is 30, and the lowest possible score is 6.The third sub-dimension was named 'Use of Media Tools'-(Use), which was measured by items Q15, Q22, Q18, and Q16.The item Q22 of this dimension was reverse coded.The highest possible score from this dimension is 20, and the lowest possible score is 4. The highest possible score on the entire scale is 80, and the lowest possible score is 16.

CONCLUSION
The Media Literacy Perception Scale was developed in this study, and validity and reliability tests were carried out.The Media Literacy Perception Scale has sixteen items and three factors.
In the current study, the Cronbach's alpha coefficient for the entire scale was .86; the dimension of the media's effect on society was .82; the dimension of the metacognitive media awareness was .74;and dimension of the media tool use was After considering the analyses conducted, it is possible to conclude that the developed scale has validity and reliability characteristics that can measure media literacy perceptions.The scale can be applied to groups of students other than university students, as well as to various segments of society, and qualitative studies and mixed-pattern studies may also be conducted in parallel with this study.

Table 3 .
Goodness of Fit Values for the First and Second Level CFA