Al Gold

This conversation is closed.

Should Google and Bing be invited to play Jeopardy against Watson.

Now that a computer has been matched against humans, should computers go head to head to further advance the technology? The rules should be flexible so that innovation can be introduced by each contestant.

  • thumb
    Feb 19 2011: Absolutely. I think Watson will not be successful, because it should be updated manually. Here is one blog post about using Wikipedia to answer Jeopardy! questions:

    http://memesteading.com/2011/02/16/ibm-watson-overprovisioned-big-iron/
    • thumb
      Feb 19 2011: I don't think people realize how much brute force search and match was involved. Watson cannot be considered a thinking entity.
      • thumb
        Feb 19 2011: Could it be fair to say "I don't think people realize how much 'mental' brute force and match was involved" by the human participants?

        Kudos to all involved be they hardware, software or wetware. I suspect this will be looked at from the future as an important event.
  • Feb 19 2011: Yes, and not just Google and Bing – any team! It should be open like the DARPA Grand Challenges, or Netflix Prize.

    Apparently at the outset of the idea to have IBM's Q&A technology play Jeopardy, IBM ran a short "basement baseline" test with one engineer, one month, one machine – but allowed to use the net. It's described in this excerpt from Stephen Baker's 'Final Jeopardy' book about the Watson project:

    http://ca.gizmodo.com/5759014/could-watson-have-been-defeated-by-homebrew

    In tests, the homebrew was “nearly matching” Piquant, IBM’s at-the-time “state-of-the-art in Q-A technology”, though both fell far short of being competitive with top humans. In the excerpt, that's spun as confirming that a large multi-year giant-cluster project would be necessary.... but my thought is: if a fresh graduate with little time, no collaborators, and few resources could match something with far more investment, give the scrappy, stingy, homebrew tactics some more attention! And a few more people, machines, and months (if not years).

    That's what a project would have done if its foremost goal was competing on Jeopardy, efficiently. It looks like the Watson project had lots of other competing promotional goals for IBM. That's OK – the prospect of eventually selling roomfuls of computers and multi-million-dollar software packages in commercial domains, domains that are even more challenging than formulaic trivia contests, is what made the work possible.

    But it means we haven't really tested, in a competitive sense, how clever and compact a trivia-dominating system could be. For example, this process as run by IBM and Jeopardy is nothing like the years of work by many teams on Chess-competitive computer systems, and independent man-machine contests, which gave confidence that chess-champion 'Deep Blue' was really a new high-water mark in computer potential.

    Only multiple competing efforts could provide that confidence.
  • Feb 19 2011: I think that this idea seriously underestimates the difficulty of processing the natural language clues. Of course you can use Google or Bing to find the answers, but I would guess that just feeding Google or Bing the clue would produce very few answers. And remember that you have to respond with the required question, not just the name of a document that contains the answer.
    • thumb
      Feb 20 2011: Of course Google , Bing, and any other challenger would need a lot of programming to analyze the English languge and the confusing clues. However, the main thrust of the contest is to search for matches and find the one best suited to the question. A search engine already has a head start.
  • thumb
    Feb 18 2011: I would absolutely LOVE to see that! I am a little bias towards IBM so I will always be rooting for him/them.