Exploiting semAntic and social knowledGe for visuaL rEcognition

By , October 2, 2015

Marie Curie IOF – European Commission FP7 People – Grant No. 623930
From 2014-10-01 to 2017-09-30

Marie Curie ActionsPeople are now more connected than ever before thanks to the technological advances of computers, smart personal devices, as well as global Internet connections. Web-based services like Flickr and YouTube and social networks such as Facebook have become more and more popular, allowing people to easily upload, share and annotate personal media content. Although visual data are the largest component of the digital world, they have been so far excluded from most social media research.

The aim of EAGLE is to exploit semantic and social knowledge for visual recognition. Our work goes beyond previous studies by introducing a new framework which jointly models objective knowledge, given by the visual appearance, prior knowledge, given by visual and linguistic ontologies, and noisy collective knowledge, given by social tags and social relations. This is a challenging issue since, at the scale of web, we are looking at connections of thousands of elements.

Outreach Activities

During the summers, I have also been involved as computer vision project mentor and research instructor in the Stanford Artificial Intelligence Laboratory’s OutReach Summer program (SAILORS). SAILORS is a two-week technically rigorous camp for high school girls and it is intended to increase diversity in the field of AI.

Dissemination Activities and Media Coverage

On November 11, 2016, I have been selected as MSCA Fellow of the week. See the post on Facebook: https://t.co/KUPmxG7E7F.

Publications

  • L. Ballan, F. Castaldo, A. Alahi, F. Palmieri, and S. Savarese, "Knowledge Transfer for Scene-specific Motion Prediction," in Proc. of European Conference on Computer Vision (ECCV), Amsterdam, Netherlands, 2016.
    @inproceedings{eccv16,
      author = {Ballan, Lamberto and Castaldo, Francesco and Alahi, Alexandre and Palmieri, Francesco and Savarese, Silvio},
      title = {Knowledge Transfer for Scene-specific Motion Prediction},
      booktitle = {Proc. of European Conference on Computer Vision (ECCV)},
      address = {Amsterdam, Netherlands},
      month = {October},
      doi = {10.1007/978-3-319-46448-0_42},
      url = {http://arxiv.org/abs/1603.06987},
      poster = {http://www.lambertoballan.net/downloads/2016_eccv_poster.pdf},
      reviews = {http://www.lambertoballan.net/downloads/2016_ECCV_213_REVIEWS.txt},
      icon = {http://www.lambertoballan.net/images/eccv_2016.jpg},
      year = {2016}
    }
  • X. Li*, T. Uricchio*, L. Ballan, M. Bertini, C. Snoek, and A. Del Bimbo, "Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval," ACM Comp. Surveys, vol. 49, iss. 1, p. 14:1-14:39, 2016. [IF: 5.243]
    @article{csur15,
      author = {Li*, Xirong and Uricchio*, Tiberio and Ballan, Lamberto and Bertini, Marco and Snoek, Cees and Del Bimbo, Alberto},
      title = {Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval},
      journal = {ACM Comp. Surveys},
      publisher = {ACM},
      volume = {49},
      number = {1},
      pages = {14:1--14:39},
      month = {Jun.},
      doi = {10.1145/2906152},
      url = {http://arxiv.org/abs/1503.08248},
      impact = {[IF: 5.243]},
      software = {https://github.com/li-xirong/jingwei},
      cited = {http://scholar.google.com/scholar?cites=3937466750334492954},
      icon = {http://www.lambertoballan.net/images/csur.jpg},
      year = {2016}
    }
  • J. Johnson*, L. Ballan*, and L. Fei-Fei, "Love Thy Neighbors: Image Annotation by Exploiting Image Metadata," in Proc. of IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015. (* equal contribution) (Poster, Accept Rate 30%)
    @inproceedings{iccv15,
      author = {Johnson*, Justin and Ballan*, Lamberto and Fei-Fei, Li},
      title = {Love Thy Neighbors: Image Annotation by Exploiting Image Metadata},
      booktitle = {Proc. of IEEE International Conference on Computer Vision (ICCV)},
      address = {Santiago, Chile},
      month = {December},
      note = {(* equal contribution) (Poster, Accept Rate 30%)},
      doi ={10.1109/ICCV.2015.525},
      url = {http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Johnson_Love_Thy_Neighbors_ICCV_2015_paper.pdf},
      poster = {http://www.lambertoballan.net/downloads/2015_iccv_poster.pdf},
      video = {https://www.youtube.com/watch?v=JW0aNtCzDks},
      reviews = {http://www.lambertoballan.net/downloads/2015_ICCV_26_REVIEWS.txt},
      cited = {http://scholar.google.com/scholar?cites=10233997843980298556},
      icon = {http://www.lambertoballan.net/images/iccv_2015.jpg},
      year = {2015}
    }
  • L. Ballan, M. Bertini, G. Serra, and A. Del Bimbo, "A Data-Driven Approach for Tag Refinement and Localization in Web Videos," Computer Vision and Image Understanding, vol. 140, pp. 58-67, 2015. [IF: 2.134]
    @article{cviu15,
      author = {Ballan, Lamberto and Bertini, Marco and Serra, Giuseppe and Del Bimbo, Alberto},
      title = {A Data-Driven Approach for Tag Refinement and Localization in Web Videos},
      journal = {Computer Vision and Image Understanding},
      publisher = {Elsevier},
      volume = {140},
      pages = {58--67},
      month = {Nov.},
      doi = {10.1016/j.cviu.2015.05.009},
      url = {http://arxiv.org/abs/1407.0623},
      impact = {[IF: 2.134]},
      reviews = {http://www.lambertoballan.net/downloads/2015_CVIU_REVIEWS.txt},
      cited = {http://scholar.google.com/scholar?cites=13219841874483285369},
      icon = {http://www.lambertoballan.net/images/cviu.gif},
      year = {2015}
    }

Tutorials, Demos & Benchmarks

  • X. Li*, T. Uricchio*, L. Ballan, M. Bertini, C. G. M. Snoek, and A. Del Bimbo, "Image Tag Assignment, Refinement and Retrieval," in Proc. of ACM International Conference on Multimedia (ACM-MM), Brisbane, Australia, 2015. (* equal contribution) (Tutorial)
    @inproceedings{mm15,
      author = {Li*, Xirong and Uricchio*, Tiberio and Ballan, Lamberto and Bertini, Marco and Snoek, Cees G. M. and Del Bimbo, Alberto},
      title = {Image Tag Assignment, Refinement and Retrieval},
      booktitle = {Proc. of ACM International Conference on Multimedia (ACM-MM)},
      address = {Brisbane, Australia},
      month = {Oct},
      publisher = {ACM Press},
      software = {https://github.com/li-xirong/jingwei},
      url = {http://www.micc.unifi.it/publications/2015/LUBBSD15/2015_mm_xirong.pdf},
      note = {(* equal contribution) (Tutorial)},
      icon = {http://www.lambertoballan.net/images/icon.jpg},
      year = {2015}
    }

Comments are closed

Panorama Theme by Themocracy