It is very difficult to distinguish gibberish essays from grammatically ill formed essays. There is no clear line of separation between these two kinds of essay. While a grammatically ill formed essay can be considered as a genuine attempt by a student whose written English is not very good, the gibberish essay when considered as word salad, is a deliberate and conscious attempt by a student to cheat the system. Technically a gibberish essay is not always a result of student trying to trick the system. An essay written by a student who's English is poor can turn out to be gibberish too. For a human reader it is quite easy to differentiate between gibberish and a grammatically poor essay. This is so because we can extract the intended meaning from a phrase with poor grammar but we cannot extract the intended meaning from the gibberish phrase. On the other hand it is not very straight forward for the automated essay grading system to perform the same task since it does derive its judgement based on semantics of the content. It has to follow predefined rules which act differently in different context, hence contributing to erroneous results in several cases. Example of a gibberish essay: Question: Euthanasia is a very controversial topic. There are people who think that mercy killing should be allowed to relieve people from endless pain and there is a group of people who think that we should not play god. Express your opinion on the issue. Answer: "Mercy pong who ping mercy. life why god life. Murder Human king sick right people is kick endless. pain right die, We god are god. when Respect life how. Euthanasia tired bad is. All Murder murder murder murder murder. No right to kill kill kill. Pain god god god god god god god god god Mercy killing. life is god. Murder Human rights sick people endless. sick Right die, We are not god. Respect for life. Euthenesia is bad. Natural death. Murder murder murder murder murder. No right to kill kill kill. Pain god god god god god god god god god." Our gibberish detection modules will detect a wide variety of gibberish contents from the essay. They might miss the very subtle cases, but they will detect all other explicit and basic cases. 1) The ratio based gibberish detection module will detect the essays with word salad in the case where the nouns will be too many or too few. (More details in the implementation section) 2) The rule based detection module will detect the portions of essay that contradict the rules of grammar. If there are many sentences that contradict the laws of grammar then this module considers that essay to be gibberish. (More details in the implementation section) The linear combination of the scores form these modules are used to determine the final degree of gibberishes. (Analysis of each score and combined score is presented in the implementation section) Irrelevance can be defined as something that is not related to the matter being discussed. Irrelevance detection deals with identification of essays that are not relevant to the subject and measures how much the essay goes along with the topic. Irrelevance also measures whether the student has understood what the essay topic means and what is expected as an answer. This gives a good insight on the capabilities of the student. The ideal answer should be prompt specific and must discuss all the issues involved without any deviation from the topic. Presence of irrelevant contents in the essay reveals that the student is unaware of what is required. A drift from the main issue at hand, and inclusion of issues that are not related to the question, suggests that the student is not well informed about the issue or he/she is not creative enough to relate the issue to something they have experienced and make an argument based on that. Irrelevance is the most common fault in essays and can be attributed to lack of knowledge on the issue. An essay created by a student in a perfectly grammatical form and presented in a very elegant manner should be considered as a poor essay if it lacks the relevant contents. Another passage: "Foo boo to moo and black sofa invented space cabbage cabbage in zoo moo poo hoo hoo. what is what and what is what is what? do you what is know meow hoo hoo. when cabbage eats cabbage then it eats what eats what? today is meow and tomoro is hohohohohoh. why is kobayashi kobayashiand meow is benzene. when cabbage went weekly to monthly to moon is when picnic went kuku na paka. by the way cutting is cutting and utting is cutting. kantai can tai a tai and untie a tai why cant i tie a tai and untie a tai as kantai can tie a tai and untie a tie. he ran and ran and ran and ran till cabbage caught the baseball in the baseball. why are we cabbaging the cabbage with new and old bus in yahoo and yahoo? we should kick the kick with kick and a kick but kicks and cabbage is cabaage kick with cutting the cheese. When hary met sally and sally met kantai, but kantai can tie a tie and untie a tie but why can't i tie a tie and untie a tie as kantai can tie a tie and untie a tie. god knows why." Observation: This essay is gibberish semantically rather than syntactically. Module 1 suffers badly since the ratio of nouns to (determiners + conjunctions + verbs + adjectives) is within 0.5 and 4.0. This is so because the number of nouns does not stand out from the rest of the tags as it did in test 1 & 2. Module 2 & 3 do account for the gibberish sentences but module 4 provides the ideal result that is the essay is around 90% gibberish. Unfortunately the scoring mechanism does consider input from module 4 since the difference of scores between module 1 and the average of 2 & 3 is lower than module 1 & 4 or module 1 and the average of module 2 & 3. Another passage: "Repeating oneself consatantly can also
come from doubts, unreliability & suspicions about one' own beliefs/religion
& repitition is used to re-enforce such doubts & sane people should
always have doubts about 'fairy-tale' type stuff which Islam
has plenty of & the more fairy-tale-ish, the more do Fundos believe
in them fanatically. Secondly, it's no use trying to convert daily already who
are muslims...........you are only boring them & demonstrating your
doubts thru repeating. Besides, automatons/robots,
zombies chant/repeat mindlessly, not
humans...............
These are symptoms of the same
mis-guided people who actually thought TSUNAMI was an act of God /Allah to
punish sinners!! God loves tourism, he's on every Cruise
ship."
Observation: ???
*************************************************************************** {Invite (mankind, O Muhammad ) to the Way of your Lord (i.e. Islam) with wisdom (i.e. with the Divine Inspiration and the Qur'an) and fair preaching, and argue with them in a way that is better. Truly, your Lord knows best who has gone astray from His Path, and He is the Best Aware of those who are guided.} (Holy Quran-16:125) {And who is better in speech than he who [says: "My Lord is Allah (believes in His Oneness)," and then stands straight (acts upon His Order), and] invites (men) to Allah's (Islamic Monotheism), and does righteous deeds, and says: "I am one of the Muslims."} (Holy Quran-41:33) The prophet (peace and blessings of Allah be upon him) said: "By Allah, if Allah guides one person by you, it is better for you than the best types of camels." [al-Bukhaaree, Muslim] The prophet (peace and blessings of Allah be upon him) also said, "Whoever calls to guidance will have a reward similar to the reward of the one who follows him, without the reward of either of them being lessened at all." [Muslim, Ahmad, Aboo Daawood, an-Nasaa'ee, at-Tirmidhee, Ibn Maajah] -------------------------------------------------------------------------- All views expressed herein belong to the individuals concerned and do not in any way reflect the official views of IslamCity unless sanctioned or approved otherwise. If your mailbox clogged with mails from IslamCity, you may wish to get a daily digest of emails by logging-on to http://www.yahoogroups.com to change your mail delivery settings or email the moderators at [EMAIL PROTECTED] with the title "change to daily digest". Yahoo! Groups Links
|
