Final Lecture: Infinite Relational Model

Jesse Mu

2016-11-02

1 The Infinite Relational Model

What happens when you want to cluster your data, but the number of clusters is unknown? While some approaches involve fitting several models with a varying number of clusters to your data and comparing model fit statistics, the Bayesian approach is to specify a model which is allowed to dynamically grow the number of clusters as the complexity of the data warrants.

The Infinite Relational Model is the prototypical example of such a model (Kemp et al. 2006).

For this last project, a simple version of Charles Kemp’s Infinite Relational Model (IRM) was coded in irm.R to co-cluster rows and columns of a simple 2-dimensional binary relation.

1.1 Demo

source('irm.R')

Sanity check

As a sanity check, the toy matrix from the original paper is used:

R = rbind(
  c(0, 0, 1, 0, 1, 0, 0, 1, 0),
  c(0, 0, 0, 0, 0, 0, 1, 0, 1),
  c(0, 0, 1, 0, 0, 0, 1, 0, 1),
  c(0, 1, 1, 0, 0, 0, 0, 1, 1),
  c(0, 0, 0, 0, 0, 0, 1, 0, 1),
  c(0, 1, 1, 0, 1, 0, 0, 1, 0),
  c(1, 0, 0, 0, 0, 1, 0, 0, 0),
  c(0, 0, 0, 0, 0, 1, 1, 0, 1),
  c(1, 0, 0, 1, 0, 1, 0, 0, 0)
)
plot.R(R)

Z = irm(R, sweeps = 1000)
top.n(Z)
#> 122121323 122123424 122121324 122121343 122321424 123121424 122131425 122324525 
#>       913        23        14        13        12        10         4         4 
#> 123121434 123131434 
#>         3         2
plot.R(R, mode.irm(Z))

This successfully finds the clusters of rows and columns that correspond to the original paper.

Kemp, Charles, Joshua B. Tenenbaum, Thomas L. Griffiths, Takeshi Yamada, and Naonori Ueda. 2006. “Learning Systems of Concepts with an Infinite Relational Model.” Proceedings of the National Conference on Artificial Intelligence (AAAI) 21: 381–88. http://web.mit.edu/cocosci/archive/Papers/Kemp-etal-AAAI06.pdf.