The world’s Largest Sharp Brain Virtual Experts Marketplace Just a click Away
Levels Tought:
Elementary,Middle School,High School,College,University,PHD
| Teaching Since: | Apr 2017 |
| Last Sign in: | 103 Weeks Ago, 4 Days Ago |
| Questions Answered: | 4870 |
| Tutorials Posted: | 4863 |
MBA IT, Mater in Science and Technology
Devry
Jul-1996 - Jul-2000
Professor
Devry University
Mar-2010 - Oct-2016
Read the attached research paper about : "Human Intelligence Needs Artificial Intelligence".
Write a Summary report using APA style and include a conclusion that has your own opinion about the topic.
Human IntelligenceNeedsArtificial IntelligenceDaniel S. WeldMausamPeng DaiDept of Computer Science and EngineeringUniversity of WashingtonSeattle, WA-98195{weld,mausam,daipeng}@cs.washington.eduAbstractCrowdsourcing platforms, such as Amazon Mechanical Turk,have enabled the construction of scalable applications fortasks ranging from product categorization and photo taggingto audio transcription and translation. These vertical appli-cations are typically realized with complex, self-managingworkflows that guarantee quality results. But constructingsuch workflows is challenging, with a huge number of alter-native decisions for the designer to consider.We argue the thesis that “Artificial intelligence methods cangreatly simplify the process of creating and managing com-plex crowdsourced workflows.” We present the design ofCLOWDER, which uses machine learning to continually re-fine models of worker performance and task difficulty. Us-ing these models, CLOWDER uses decision-theoretic opti-mization to 1) choose between alternative workflows, 2) opti-mize parameters for a workflow, 3) create personalized inter-faces for individual workers, and 4) dynamically control theworkflow. Preliminary experience suggests that these opti-mized workflows are significantly more economical (and re-turn higher quality output) than those generated by humans.IntroductionCrowd-sourcing marketplaces, such as Amazon Mechani-cal Turk, have the potential to allow rapid construction ofcomplex applications which mix human computation withAI and other automated techniques. Example applicationsalready span the range from product categorization [2],photo tagging [24], business listing verifications [16] to au-dio/video transcription [17; 23], proofreading [19] and trans-lation [20].In order to guarantee quality results from potentiallyerror-prone workers, most applications use complex, self-managing workflows with independent production and re-view stages. For example, iterative improvement [14] andfind-fix-verify workflows [1] are popular patterns. But de-vising these patterns and adapting them to a new task is bothcomplex and time consuming. Existing development envi-ronments,e.g.Turkit [14] simplify important issues, suchas control flow and debugging, but many challenges remain.For example, in order to craft an effective application, thedesigner must:•Choose between alternative workflows for accomplish-ing the task.For example, given the task of transcribingan MP3 file, one could ask a worker to do the transcrip-tion, or first use speech recognition and then ask work-ers to find and fix errors. Depending on the accuracyand costs associated with these primitive steps, one or theother workflow may be preferable.•Optimize the parameters for a selected workflow.Sup-pose one has selected the workflow which uses a singleworker to directly transcribe the file; before one can startexecution, one must determine the value of continuous pa-rameters, such as the price, the length of the audio file,etc.. If the audio track is cut into snippets which are toolong, then transcription speed may fall, since workers of-ten prefer short jobs. But if the audio track is cut intomany short files, then accuracy may fall because of lostcontext for the human workers. A computer can method-ically try different parameter values to find the best.•Create tuned interfaces for the expected workers.Theprecise wording, layout and even color of an interface candramatically affect the performance of users. One can useFitt’s Law or alternative cost models to automatically de-sign effective interfaces [7]. Comprehensive “A-B” test-ing of alternative designs, automated by computer, is alsoessential [12].•Control execution of the final workflow.Some deci-sions, for example the number of cycles in an iterativeimprovement workflow and the number of voters usedfor verification, can not be optimally determineda priori.Instead, decision-theoretic methods, which incorporate amodel of worker accuracy, can dramatically improve onnaive strategies such as majority vote [3].Our long-term goal is to prove the value of AI methodson these problems and to build intelligent tools that fa-cilitate the rapid construction of effective crowd-sourcedworkflows. Our first system, TURKONTROL [3; 4], used apartially-observable Markov decision process (POMDP) toperform decision-theoretic optimization of iterative, crowd-sourced workflows. This paper presents the design of oursecond system, CLOWDER1, which we are just starting toimplement. We start by summarizing the high-level archi-tecture of CLOWDER. Subsequent sections detail the AI rea-1It is said that nothing is as difficult as herding cats, but maybedecision theory is up to the task? Aclowderis a group of cats.
-----------