Lately there is lot of research interest in the area of the benchmarking of Graph Databases and NOSQL, which is a great area to explore. However, it is also of relevance to take a look at research that is not about Graph Databases per se but instead uses some of their advantages to perform the experiments.
This is the case we would like to share in this article; a master thesis directed by DAMA-UPC that proposes a new metric to automatically evaluate the relationship between social endogamy and the quality of conferences.
The idea of the metric raises of the observation that there is a strong social aspect in conference publishing as people meet experts in their field of research and interact with them. In addition current metrics to rank the quality of conferences seem to ignore this fact:
- Paper citations: The quality of a conference is computed considering the number of citations of its papers. There are different approaches for this metric such as the H-index, the impact factor or the Page Rank. One of the main disadvantages of this metric is that, for instance, it will wrongly rank new papers which may be of a high quality but are still not citied by any other one.
- Manual ranking: Some organizations, like Excellence in Research for Australia (ERA), manually rank the conferences considering the quality of the contents.
The new metric uses endogamy, described as the inclination of a person to collaborate within a small group of selected people, as a predictive key of the quality of research. The experimental environment to evaluate the efficiency of the new metric contained the DBLP database as the dataset, which was stored in a DEX graph database* and the results of ERA of 2010 as a comparative metric. The graph databases capacities to analyze social relationships are of good use in this type of experiments.
The results of the new metric are compared to those of ERA obtaining an 85% of agreement. They show that:
- Papers published in highly reputed conferences were published by people with low endogamy.
- Conferences where researchers work with new peers have more quality than those of stable groups.
- Quality of the previous collaborations may also be taken in consideration in the metric.
For more details about the experiments, conclusions and future work we recommend reading the complete thesis here .
* DEX high-peformance graph database was used for the experiments. One of its main characteristics is its query performance for the retrieval and exploration of large networks. Its implementation with very light specialized structures allows analyzing and querying billions of objects at very low storage cost.
-----DAMA-UPC is part of the Computer Architecture Department (DAC) at Barcelona Tech . The work of the group focuses on the research and technology transfer in topics related to the management and retrieval of large data volumes, information quality and data exploration. DAMA-UPC is member of Tecnio since 2005 for its excellence of research & technology transfer.