Remote Usability Assessment of Topic Visualization Interfaces with Public Participation Data: A Case Study

Citizen participation often faces challenges of transparency and accountability. Visualizations’s usability becomes key for public consultation activities. The tree map is frequently used to disseminate data and to give it back to the population. The purpose of this study is to understand how tree maps and stacked barcharts differ in terms of effectiveness, efficiency, and satisfaction in tasks, like solving topic categorization and comparison analysis tasks. An experimental design was used to examine user performance based on a task-based usability test. 34 participants interacted remotely with data visualizations from an open 2016 participatory constitution-making process. The ANOVA showed that stacked barcharts work significantly better for comparison tasks than the tree map, but there are no significant differences in regards to categorization tasks. Public participation initiatives should first determine what cognitive operations their users perform before deciding which visualization interfaces will be more useful for the intended public.


Introduction
'Citizen participation' is a broad category that describes citizen involvement in public decision making (Irvin & Stansbury, 2004). Depending on the conceptual framework you sit upon, 'citizens' may either be individuals or organized communities, and 'participation' may involve either observation or direct power (Baum, 2015). Additionally, the scope of participation can be at the community or national level and in various spheres of government and society (Cardullo & Kitchin, 2019). One of the earliest definitions of citizen engagement was proposed by Arnstein (1969). The author Different visualization interfaces have been developed to represent the topic models (Karpovich, Smirnov, Teslya & Grigorev, 2017). The aim of those interfaces is to allow the user to analyze topic models. The tree map is one of the most used visualizations to present an overview of topics organization and allow categorical and quantitative analysis (Sukhija et al., 2019); how many topics, what are more important, and what are the categories between them. The biggest problem with tree maps is that using size to encode the data makes it difficult to make precise quantitative comparisons versus using length of a bar . In that sense, other visualization strategies such as stacked bar charts, may offer a better way of encoding data for precise comparisons.
This article explores and compares usability characteristics of two types of topic visualization interfaces using data from a public participation exercise. We compare the usability of the stacked bar chart and the tree map visualization techniques within an interface complemented by two maps: a choropleth map and a point density map, since the use of maps has previous evidence of its validity in supporting georeferenced information in analysis tasks (Jankowski & Nyerges, 2001). For the topic model, the Citizen Consultation of the 2016 Constituent Process in Chile was utilized as a case study. This particular consultation brought together more than 200,000 citizens who gave their opinion for the content of a possible new constitution (OECD, 2017). This inquiry is based on the interest in the design of interfaces in the field of human-computer interaction (HCI) as described by researchers such as Polasanapalli and Buggareddy (2020).

Topic modeling and topic visualization
With the rise of computer science, text analysis tasks have become automated (Humphreys & Wang, 2018). Artificial intelligence (AI) is the scientific field for it. With a large amount of training data, AI allows the understanding of different perspectives that underlie documents (Radford, Narasimhan, Salimans & Sutskever, 2018). One of the most common and useful perspectives for understanding documents is through their main topics. The process of learning and extracting these topics is called topic modeling (Brynjolfsson & McAfee, 2017). This machine learning technique applies probabilistic models to extract underlying themes from large collections of documents. (Blei, Ng & Jordan, 2003). In topic modeling research, a topic is defined as a set of words that has a high probability of appearing together (Jelodar et al., 2019).
Latent Dirichlet Assignment (LDA) is one of the most popular methods in topic modeling (Sukhija et al., 2019). LDA has been used for opinion modeling from social media (Parra et al., 2016;Sun, Luo & Chen, 2017;Xu, Liu, Xuan, Chen & Mei, 2017), customer reviews (Calheiros, Moro & Rita, 2017) and public consultations (Hagen, Uzuner, Kotfila, Harrison & Lamanna, 2015). Although it has been useful for experts in the area of computer science, the results of the model are not very intuitive for ordinary users (Cai, Sun & Sha, 2018). That's why in recent years, several topic visualization interfaces have been developed to simplify and communicate what people opine in social media, about products and services or in public affairs. (Karpovich, Smirnov, Teslya & Grigorev, 2017). One of the main objectives of this type of visualization is to provide users the options to explore the set of documents and identify what topics they relate to (Ganesan, Brantley, Pan & Chen, 2015).
According to Yang & Qu (2017) there are three guidelines for building efficient and user-centered topic visualization interfaces (i) An overview of the organization of the topics that allows the user to understand how many topics there are, what is more important and what are the categories, (ii) a geographic perspective view of the topics that allows the user to explore geolocated data; (iii) a particular view of opinions by topic that allows the user to review the compilation of opinions or an extract from them. Different methodologies have emerged to follow these guidelines and create user-centered visualizations. One of them is the one proposed by Tamara Munzner (2009). This method combines four areas of knowledge: anthropology/ethnography, design, computer science, and cognitive psychology. Munzner proposes designing visualizations through three fundamental questions that are asked iteratively. These questions are: 1) What information will be displayed on the visualization? 2) Why does the user need the information?
3) How will the information be encoded?
To design topic visualization interfaces, questions 1 and 2 can be answered through the guidelines previously outlined. For question 3, Mackinlay (1986) suggests coding the attributes with the best-ranked channels according to the principle of effectiveness.

Visual encoding for topic visualizations
Data representation is a main component of information visualizations. The fundamental focus of data representation is mapping from data values to graphical representations (Cleveland & McGill, 1985). Visualization designers use elementary graphical units called visual encodings to translate data to visual elements (Kim & Heer, 2018). There are different visualization alternatives to encode the same information. It is important that these alternatives meet expressiveness criteria (Mackinlay, 1986). Given two graphical languages that express some information, the obvious question is which language involves a design that specifies the more effective presentation. Unlike expressiveness, which only depends on the syntax and semantics of the graphical language, effectiveness also depends on the capabilities of the users's personal characteristics (Qu & Hullman, 2016), such as cultural background.
Among the most popular visualizations to present an overview of topics distribution and categorization is the tree map (Long, Hui, Fook & Zainon, 2017). In it, the importance of each topic over the total quantity is encoded by the area of each square. The larger the area of the square, the greater the importance of the topic in question (Vernier, Telea & Comba, 2018). An alternative for this type of data is stacked barcharts, where the importance of each topic over the total quantity is encoded through the position and the length of the bar (Howorko, Boedianto & Daniel, 2018). In both cases, tree map or stacked barchart, the categories of the topics are encoded through color tones.
Several researchers have examined empirical user performance with visual encodings (Ware, 2019), validating and refining effectiveness rankings originally proposed by Bertin (1983). Cleveland & McGill (1985) conducted human-subjects experiments measuring decoding error across encoding channels, for example finding that position encodings outperform length encodings when comparing proportions. In addition to studies of univariate encodings, researchers have examined interactions between visual encoding channels (Garner & Felfoldy, 1970;Ware, 2019). For example, integral visual channels (e.g., color and size) may facilitate decoding when used to redundantly encode the same field (Demiralp, Bernstein & Heer, 2014).
A more recent trend is the study of tasks involving the perception of visual information. A large body of earlier work (Correll & Gleicher, 2014;Szafir, Haroz, Gleicher & Franconeri, 2016;Dimara, Bezerianos & Dragicevic, 2017) has studied the effectiveness of visualization types with common design configurations for a selected number of tasks. These studies usually focus on specified users, the specific ones for whom the visualization is designed, and on the specified context of use and the precise environment in which the users will use it (Barnum, 2020). Usability evaluation is the method that has been used to measure the encompass of tasks, users and context (Kim & Heer, 2018).

Usability as an evaluation method of topic visualization interfaces
Within the field of human-computer interaction, usability has become a highly used concept when evaluating visualizations (Umar, Bakhat and Hassan, 2020). Usability has become part of the set of standards established by the International Organization for Standardization (ISO). Usability is understood as the degree to which a product can be used by specific users to achieve goals with effectiveness, efficiency and perceived satisfaction in a specific context of use (ISO 9241). Therefore usability is understood and measured in three dimensions: 1) Effectiveness refers to the performance of tasks, to the precision with which the user meets their objectives. It is usually evaluated based on precision such as error rate or success rate (Howorko et. Al, 2018). 2) Efficiency refers to the relationship between the level of effectiveness and the consumption of resources. For example, metrics of the number of clicks and time to complete the task are used (Zia, Guo, Zhou, Essa & Jarc, 2019). 3) Perceived satisfaction refers to the comfort reported by the user after solving a task. Satisfaction level metrics are used on Likert scales (Isaac, Abdullah, Ramayah & Mutahar, 2017).
Usability evaluation methods are used to evaluate the interaction of the human with the computer for the purpose of identifying aspects of this interaction that can be improved to increase usability (Lewis, 2019). Through usability evaluations, the most appropriate elements for an interface are identified. In them, the degree of effectiveness, efficiency and satisfaction with which users can use the system designed to solve certain tasks is measured (Manresa-Yee, Amengual, & Ponsa Asensio, 2014).
The main goal of topic model data analysis is to identify topic clusters, explore patterns and locate representative opinions of these topics. The topic model analysis process can be viewed as a set of tasks and operations needed to meet the goals of the data exploration. These tasks involve a number of more specific activities and operations that users will perform such as (Karpovich, Smirnov, Teslya & Grigorev, 2017): 1) Identification of how topics are categorized.
2) Identification of which category a certain opinion belongs to.
3) Comparison of the relative importance between topics. 4) Comparison of relative importance of a certain topic at different spatial locations.
To complete the tasks described above, the user will have to execute a number of visualization operations during the exploration process. Several authors have suggested taxonomies for visualization operations (Qian, Wachowicz, Peuquet & MacEachren, 1997;Zhou & Feiner, 1998;Ogao & Kraak, 2002). The most comprehensive list (Keller & Keller, 1992;Wehrend & Lewis, 2000) includes identify, locate, categorize and compare: 1) Identify: to establish the collective characteristics by which an object is distinctly recognizable. 2) Locate: to determine the absolute or relative position. 3) Categorize: to place in specifically defined divisions in a classification; this can be done by colour, position, or type of object (shape). 4) Compare: to examine so as to notice similarities, differences, order.
In conclusion, several advances have been made in modeling and visualizing public opinion. However, recent studies tend to focus mostly on social media opinion analysis (Steinskog, Therkelsen & and Gambäck, 2017;Yu, Xu, Wang & Ni, 2019;Kant, Weisser & Säfken, 2020). The modeling and visualization of opinions from institutional citizen participation' instances is still missing more attention (Mukherjee, 2019). That way, it would become more manageable to address the growing Creative Commons Attribution 3.0 Austria (CC BY 3.0), 2021.
interest and demand towards a more active, direct and substantial public participation in policymaking (Dryzek, 2002;Floridia, 2018).
On the other hand, usability evaluation encompasses three dimensions: effectiveness, efficiency and satisfaction. But since the beginning of visualization evaluation, mostly the effectiveness has been measured (Szafir, Haroz, Gleicher & Franconeri, 2016). Moreover, despite the fact that satisfaction was important when the ISO standard was developed, it has just recently begun to be considered (Barnum, 2020). Furthermore, about half of the studies in the area do not include evaluations of any kind (Elmqvist & Yi, 2015). Usability studies are needed to evaluate and validate the relevance of visualization interfaces (Yang et al., 2017).
In sum, it is not enough to use visualization strategies. Rather, the effectiveness of public processes requires that institutions provide evidence of the relevance of these visualization tools. Our study explores usability measures in visualization interfaces for topic modeling of citizen consultations in a case of participatory constitution-making in Chile. To achieve this objective, our research question is: what are the differences in effectiveness, efficiency, and perceived satisfaction between interfaces including tree map and stacked barchart in HCI challenges?

Case study: Topic model of Chilean public consultation
In 2016, the Government of Chile convened a citizen consultation to discuss the contents of a possible new constitution. More than 200,000 citizens gave their opinion about its content (OECD, 2017). The methodology developed by the government established four lines of debate: (1) values and principles, (2) rights, (3) duties and responsibilities, and (4) State institutions (Raveau, Couyoumdjian, Fuentes-Bravo, Rodriguez-Sickert and Candia, 2020).
A guiding question was defined to address each line. For each question, a short argument had to be written to justify the answer. The content of the consultation was processed by the Systematization Committee, an autonomous body external to the Government. Its methodology consisted of a manual work of syntactic unification of each opinion, without modifying its semantics (Fierro, Fuentes, Pérez and Quezada, 2017). The results were consolidated in an open database called Bases Ciudadanas (Becker Castellaro, 2018).
The size of Bases Ciudadanas was reduced by 9.8% after cleaning empty cells. Python, NLTK Toolkit and Gensim libraries were used for preprocessing. Capitalization, accents, and punctuation have been removed through tokenization. Stop words, words such as articles, pronouns and prepositions that do not add value to the meaning of the text were removed. The words in the third person changed to the first person and the verbs in the past and future changed to the present through a stemming process.
After preprocessing, a dictionary was built with all the words present in the opinion attribute of Bases Ciudadanas in order to train the model. In that way the model could 'learn' the words and their relation with others. The number of topics (k) is an important parameter to run the model, which it has to be set beforehand. In order to set the optimal number of topics, many LDA models with different values of 'k' were run. Finally, the k's election was based on the highest coherence value (c_v). The optimal number of topics were 15 and the average training time was 26 minutes. Figure 1 shows the user interfaces for interacting with topics and opinions. Each version of the interface system has three focus areas. The upper left part is a topic distribution visualization that allows identifying the categorization of topics and the relative importance between them. The lower left part is a topic-region visualization that represents the spatial distribution of each topic. The right part is a topic-opinion visualization that allows to explore and interact with opinions in detail.

Topic visualization interfaces
For the visualization of topic distribution, the 15 topics were classified into 5 categories, where each category is represented by a different color tone. When selecting a topic, the other two visualizations are filtered according to the topic in question. The topic-region visualization changes the color saturation of each region to represent the relative importance of that topic. Also, when selecting a region, the other two visualizations are filtered based on that region. In the topic-opinion visualization each point represents an opinion and these are coded according to the color of its most representative topic. When hovering over a point on the map, a pop-up window displays the opinion, its commune and region.

Instrument
For the data collection process, a Task-Based Usability Test was used (Corry, Frick & Hansen, 1997). This kind of test is designed to explore users' performance and perceived satisfaction given specific tasks and interfaces such as categorize and compare. Setting up a usability test involves carefully creating a scenario, wherein the person performs a list of tasks using the interfaces being tested. Due to the COVID-19 health crisis, a remote usability testing was conducted (Barnum, 2020). This usability testing works well with any of the popular online meeting tools. In this case, the remote usability test was done through Zoom.
The test is structured in 4 sections covering: (1) topic categorization, (2) opinion categorization, (3) overall comparison and (4) comparison by region. The topic categorization section includes a question of alternatives: "Which topic is in the same category as "human rights"?. To answer this question two alternatives were provided. This section is focused on the use of the tree map or stacked barchart visualization.
The opinion categorization section includes a question of alternatives: "What topic could the following opinion belong to? "Creation of regional laws and regulations should enable the administration of own resources in a decentralized manner"?. To answer this question two alternatives were provided. This section is focused on the complementary use between tree map or stacked barchart, and point density map.
The overall comparison section includes a question of alternatives: "What is the most frequent topic between "New Constitution" and "Cultural Heritage"?". To answer this question two alternatives were provided. This section is focused on the use of the tree map or stacked barchart visualization.
The comparison by region section includes a question of alternatives: "In Los Lagos region, what is the most frequent topic?". To answer this question two alternatives were provided. This section is focused on the complementary use between tree map or stackedbar chart, and choropleth map.
After each section, participants had to grade 3 statements of the After Scenario Questionnaire (ASQ) (Lewis, 1995). Grading was in terms of how satisfactory it was using the interface to solve the task on a seven-point Likert scale (from 1 = "totally disagree" to 7 = "totally agree"). The statements were: 1) In general, I am satisfied with the ease of completing tasks on this interface. 2) In general, I am satisfied with the amount of time it took to complete the tasks on this interface. 3) In general, I am satisfied with the information provided to complete the tasks.

Participants
The participants in this study were a group of 34 engineers recently graduated from an university in Chile. To determine the number of participants GPower Software was used. Through a Power Analysis (Mayr, Erdfelder, Buchner & Faul, 2007), the sample size was determined assuming a significance level ( ) of .05, a statistical power (1-β) of .8 and an effect size (d) of .25.
The age of the participants was between 25 and 30 years old. It was limited to this age range due to its advantage in digital skills and to avoid effects associated with age (Hanson, 2011). The average age of the participants was 27 years old. Regarding the gender of the participants, 19 (56%) were women and 15 (44%) were men. It was considered as an exclusion criterion having a handling or experience with data visualizations greater than 5 on a scale of 1 to 10. All the participants used the same equipment and carried out the assigned tasks in a time of 30 minutes.
None of the participants received financial compensation. Participants were asked to sign a consent form and were informed that they could leave the study at any time.

Experimental design
A design of 2 types of interface and 4 types of tasks, intra-subjects design was used. Each participant used both interfaces to perform categorization and comparison tasks through the Task-Based Usability Test. The participants were first briefed about the interfaces and were clarified on the tasks that they had to perform.
Two versions were performed to counteract cognitive fatigue and to control associated learning effects (DePuy & Berger, 2014). Table 1 shows the 2x2 Latin Square used to determine the order of interface types used by participants in the study. Participants assigned to version 1 first used Interface A and then Interface B. Participants assigned to version 2 first used Interface B and then Interface A. In total, 8 questions had to be answered per participant, 4 with interface A and 4 with interface B. The metrics considered to perform the usability evaluation in this study include performance measures (task success rate, task completion time), as well as self-reported measures (perceived satisfaction) (Sørum, Andersen & Vatrapu, 2012). The success rate was measured through the number of correct answers over the total of answers per task and per interface. Completion time was measured through the time (in seconds) taken by participants to complete a specific task using a particular interface. And finally, the perceived satisfaction was measured through the comfort level reported by the participants per task and per interface. The success rate and completion time were recorded automatically each time a participant responded to a question. The perceived satisfaction was collected through the ASQ.

Data Analysis Tools
The study' aim was to explore the effects of two factors (type of interfaces and type of tasks) on success rate, completion time, and perceived satisfaction. The comparison between the two interfaces (A: Treemap + Choropleth map + Point density map versus Stacked barchart + Choropleth map + Point density map) was performed through a graphical analysis and statistical hypothesis tests.
To visually compare the results, the means and standard deviations of the performance measures across two task types and two different interface types were plotted on a bar chart. Although this type of comparison is not conclusive, it may help to identify trends on performance measures and (potential) differences between interfaces.
Given the setup of the experiment, the mean differences between the two factors were measured using two-way repeated measures analysis of variance. There were no significant outliers in any cells in the design. The dependent variable complies with the assumption of normality according to the Shapiro-Wilk test. Mauchly's test showed that the data met the assumption of sphericity. When the values were significant, post hoc comparisons with Bonferroni adjustment were made. The experimental data were analyzed with R Studio.

Results
The main objective of this study was to compare usability metrics patterns in categorization and comparison tasks by two interfaces (A: including a tree map versus B: including a stacked barchart).
Figure 2 shows the mean for success rate, completion time and perceived satisfaction for each task of the Usability Test by interface. The behavior of the usability metrics follows a similar pattern in both interfaces, in which the mean were relatively close in categorization tasks (T1 and T2), and relatively far in comparison tasks (T3 and T4). This means that there were not major differences in the success rate (see Figure 2a), completion time (see Figure 2b) and perceived satisfaction (see Figure 2c) between interfaces in categorization tasks, but there were differences between both interfaces in comparison tasks. (c) Figure 3 shows the means for success rate, completion time and perceived satisfaction for each task of the Usability Test by interface on an overlapping line chart. There is a common tendency where lines do not appear to be parallel and in some points they are even crossing. This could suggest an interaction between the factors where, success rate (see Figure 3a Graphical analysis is usually subjective, and therefore it is also necessary to use more robust statistical tools, such as hypothesis testing. Hence the two-way repeated measures analysis of the variance test was applied to compare the median of each usability metric for each interface by task. Table 2 shows the main effects of task type and interface type were qualified by a significant interaction between task type and interface type on success rate, completion time and perceived satisfaction. Due to its interaction effect, a pairwise comparison was conducted to see the simple effects between both factors on the three usability metrics.  Table 3 presents the p-values from pairwise comparison of the two-way repeated measures ANOVA with Bonferroni's correction. Corroborating and complementing the graphical analysis, comparison tasks (T3 and T4) were significantly different between interfaces in the success rate, in the task completion time and in the perceived satisfaction of the task.

Discussion
Considering the need for transparency, accountability and public understanding, visualizing public participation data without usability evidence is not enough. On the one hand, recent studies have shown the effectiveness of modeling and visualizing public opinion, but mostly on informal settings and social media (Steinskog, Therkelsen & and Gambäck, 2017;Yu, Xu, Wang & Ni, 2019;Kant, Weisser & Säfken, 2020). On the other hand, the assessment of visualization strategies have often lacked a more comprehensive operationalization of usability and focused mostly on effectiveness.
In this article we examined the usability of two different visual interfaces using data from a participatory constitution-making consultation. We sought to compare the usability of the tree map, which is the most frequently used technique, and the stacked barchart, which is conceptually more apt for quantitative analysis. A two-way repeated measures ANOVA was conducted to determine whether there were significant differences between the two interfaces in terms of success rate, completion time and perceived satisfaction and when being used for two different cognitive tasks (categorization and comparison). This approach was utilized to determine whether the usability of the two interfaces were similar or different when used for solving different tasks.
The results from the ANOVA showed that there is a significant interaction effect between task type and interface in the three dimensions of usability (see Table 2). This suggests that the interfases produce different results but only in certain tasks. In order to find out which interface produces better results in which task, a pairwise comparison was utilized (see Table 3). This analysis showed that the stacked barchart yields better results in all dimensions of usability, but only when solving tasks 3 and 4, which are comparison tasks. Tasks 1 and 2 (categorization tasks) showed no significant differences.
Our findings support the hypothesis that the stacked barchart is a better alternative to the tree map visualization for cognitive tasks that involve categorization and comparison. This may be explained by the fact that it is more difficult for the human eye to make precise comparisons when the data is encoded through the area of a figure (Few, 2009), as treemap does. On the other hand, the stacked bar chart's advantage may be supported by the hierarchy of visualisation accuracy (Mackinlay, 1986), where the use of position and length to communicate information is the most accurate way to display quantitative data. In this sense, these findings support the idea that the usability of visualization strategies is task-specific and thus, the assessment of public participation modeling and visualization should be directed at specific cognitive operations. More debate and research is needed to determine which cognitive tasks can be used as standards of success for public participation data.
In sum, the contribution of this research is twofold. Firstly, we provide evidence for the usability of two visualization techniques using data from a massive institutional participation exercise. Our results showed that the stacked barchart visualization yields better results for comparison tasks and that categorization tasks show no significant difference. Based on these results, the stacked barchart is more appropriate for visualization that involves both categorizing and comparing quantitative data. Secondly, our research indicates that the performance and usability of visualization strategies are task-specific and thus, that no generic usability assessment should be performed. In turn, this conclusion adds to the complexity of giving back information to the public as visualization strategies have to account for the expected cognitive operations of their final users.
This study focused on the quantitative approach to evaluate the usability of the interfaces. However, the experimental data did not capture the user's subjective experience in performing the tasks. It would be useful to be able to triangulate the experimental results with qualitative information to better understand how the handling of information is represented by the public and how the topic of the information is related to the cognitive task. On the other hand, the participants had a certain level of computer knowledge, since they were all recently graduated professionals. As engineering graduates, they most likely had experience programming visualizations of datasets and were familiar with elements of the task at hand. Previous knowledge and age can influence the results of the experiment. For instance, previous research has highlighted the need to consider educational gaps and digital divides in online participation (Jankowski et al., 2019) Therefore, the study's findings may not be easily generalized to non-computer users and older users. Future research should address how participants characteristics and how the topic of the information impacts the usability of visual interfaces. Finally, these experimental results were not a part of any systematic and mandatory assessment of the participation initiative described. Future studies should address how the assessment of visualizations of public engagement exercises operates as part of larger accountability policies.

Declaration of interest
The authors declare no conflict of interest.
Creative Commons Attribution 3.0 Austria (CC BY 3.0), 2021. Gender, rights and opportunities equality must be guaranteed.  What topic does the following opinion belong to? "Proactive civic relations are considered essential for the protection of human rights in our society permanently, in order to achieve nondiscrimination due to divergent, political and gender opinions"

Aprendix B
What topic does the following opinion belong to? "Creation of regional laws and regulations should enable the administration of own resources in a decentralized manner" 3 Overall comparison What is the most frequent topic between "Powers of the State" and "Environment"?
What is the most frequent topic between "New Constitution" and Cultural Heritage?

4
Comparison by region In Antofagasta region, what is the most frequent topic between "Equality before the Law" and "Participation"?
In Los Lagos region, what is the most frequent topic between "Congress" and "Transparency"? 5 Topic categorization What topic is in the same category as "Human Rights"?
What topic is in the same category as "Common Good"?

Opinion categorization
What topic does the following opinion belong to? "Creation of regional laws and regulations should enable the administration of own resources in a decentralized manner" What topic does the following opinion belong to? "Proactive civic relations are considered essential for the protection of human rights in our society permanently, in order to achieve nondiscrimination due Creative Commons Attribution 3.0 Austria (CC BY 3.0), 2021.
to divergent, political and gender opinions" 7 Overall comparison What is the most frequent topic between "New Constitution" and Cultural Heritage?
What is the most frequent topic between "Powers of the State" and "Environment"? 8 Comparison by region In Los Lagos region, what is the most frequent topic between "Congress" and "Transparency"?
In the Antofagasta region, what is the most frequent topic between "Equality before the Law" and "Participation"?