| 1 |
Evaluation in artificial intelligence: From task-oriented to ability-oriented measurement
|
|
|
|
BASE
|
|
Show details
|
|
|
| 2 |
IQ tests are not for machines, yet
|
|
|
|
In: http://www.dsic.upv.es/%7Eflip/papers/IQnotuniversal.pdf (2012)
|
|
Abstract:
Complex, but specific, tasks —such as chess or Jeopardy! — are popularly seen as milestones for artificial intelligence (AI). However, they are not appropriate for evaluating the intelligence of machines or measuring the progress in AI. Aware of this delusion, Detterman has recently raised a challenge prompting AI researchers to evaluate their artefacts against IQ tests. We agree that the philosophy behind (human) IQ tests is a much better approach to machine intelligence evaluation than these specific tasks, and also more practical and informative than the Turing test. However, we have first to recall some work on machine intelligence measurement which has shown that some IQ tests can be passed by relatively simple programs. This suggests that the challenge may not be so demanding and may just work as a sophisticated CAPTCHA, since some types of tests might be easier than others for the current state of AI. Second, we show that an alternative, formal derivation of intelligence tests for machines is possible, grounded in (algorithmic) information theory. In these tests, we have a proper mathematical definition of what is being measured. Third, we re-visit some research done in the past fifteen years in the area of machine intelligence evaluation, which suggests that some principles underlying IQ tests may require a re-visiting or even a substantial revision before using them for effectively measuring machine intelligence —since some assumptions about the subjects and their distribution no longer hold.
|
|
URL: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.225.6158
http://www.dsic.upv.es/%7Eflip/papers/IQnotuniversal.pdf
|
|
BASE
|
|
Hide details
|
|
|
| 3 |
Comparing humans and AI agents
|
|
|
|
In: http://users.dsic.upv.es/proy/anynt/paper1-comparing.pdf (2011)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Measuring Universal Intelligence: Towards an Anytime Intelligence Test
|
|
|
|
In: http://users.dsic.upv.es/proy/anynt/measuring.pdf (2010)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
A (hopefully) unbiased universal environment class for measuring intelligence of biological and artificial systems. Extended Version. available at http://users.dsic.upv.es/proy/anynt
|
|
|
|
In: http://users.dsic.upv.es/proy/anynt/unbiased.pdf (2009)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Beyond the Turing Test
|
|
|
|
In: http://users.dsic.upv.es/proy/anynt/Beyond.pdf (2000)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Beyond the Turing Test
|
|
|
|
In: http://www.dsic.upv.es/~jorallo/escrits/TT-JHdez.ps.gz (1999)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
BeyondtheTuringTest(Extended,originalinternalreport)
|
|
|
|
In: http://users.dsic.upv.es/%7Eflip/papers/Beyond2000-preliminary-extended.pdf (1999)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
The ANYNT Project Intelligence Test Λone
|
|
|
|
In: http://www.csse.monash.edu.au/~dld/Publications/2012/Insa-CabreraHernandez-OralloDoweEspanaHernandez-Lloreda_The_anYnt_ProjectIntelligenceTest:Lambda_one_AISB_IACAP_JointConference_CelebratingTheTuringYear_Birmingham_July2012_8pp.pdf
|
|
BASE
|
|
Show details
|
|
|
| 10 |
Turing Machines and Recursive Turing Tests
|
|
|
|
In: http://www.csse.monash.edu.au/~dld/Publications/2012/Hernandez-OralloInsa-CabreraDoweHibbard_TuringMachinesAndRecursiveTuringTests_AISB_IACAP_JointConference_CelebratingTheTuringYear_Birmingham_July2012_6pp.pdf
|
|
BASE
|
|
Show details
|
|
|
| 11 |
A (hopefully) Unbiased Universal Environment Class for Measuring Intelligence of Biological and Artificial Systems
|
|
|
|
In: http://agi-conf.org/2010/wp-content/uploads/2009/06/paper_19.pdf
|
|
BASE
|
|
Show details
|
|
|
| 12 |
On Discriminative Environments, Randomness, Two-part Compression and MML
|
|
|
|
In: http://www.dsic.upv.es/%7Eflip/papers/projectibility.pdf
|
|
BASE
|
|
Show details
|
|
|
| 13 |
Beyond the Turing Test
|
|
|
|
In: http://www.dsic.upv.es/%7Eflip/papers/Beyond2000.pdf
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Measuring Universal Intelligence: Towards an Anytime Intelligence Test
|
|
|
|
In: http://www.dsic.upv.es/%7Eflip/papers/measuring.pdf
|
|
BASE
|
|
Show details
|
|
|
| 15 |
Compression and intelligence: social environments and communication
|
|
|
|
In: http://users.dsic.upv.es/proy/anynt/paper5-compression.pdf
|
|
BASE
|
|
Show details
|
|
|
| 16 |
On measuring social intelligence: experiments on competition and cooperation
|
|
|
|
In: http://users.dsic.upv.es/~flip/papers/AGI2012.pdf
|
|
BASE
|
|
Show details
|
|
|
| 17 |
Turing Tests with Turing Machines
|
|
|
|
In: http://users.dsic.upv.es/%7Eflip/papers/Turing100.pdf
|
|
BASE
|
|
Show details
|
|
|
| 18 |
The ANYNT Project Intelligence Test Λone
|
|
|
|
In: http://users.dsic.upv.es/%7Eflip/papers/AISB-AICAP2012a.pdf
|
|
BASE
|
|
Show details
|
|
|
| 19 |
Turing Machines and Recursive Turing Tests
|
|
|
|
In: http://users.dsic.upv.es/%7Eflip/papers/AISB-AICAP2012b.pdf
|
|
BASE
|
|
Show details
|
|
|
|
|
