CardWorld versions 1 and 1a, May and August 2010 – Author: James A. Mason

http://www.yorku.ca/jmason/asdindex.htm

CardWorld illustrates use of ASDParser with an ASD grammar (Cardgram1.grm) to implement an application that understands English in limited ways. It is the smallest example which I have been able to construct that incorporates syntax, semantics, and pragmatics all in non-trivial ways. Thus it allows easy exploration of how those three components of language understanding interact. Also, the pragmatic domain of playing cards is rich enough that this model of language understanding can be extended in many interesting ways, syntactically, semantically and pragmatically.

Pragmatic limitations:

CardWorld permits a user to interact with images of ordinary playing cards on top of a card table. Cards and piles of cards on the table can be manipulated directly with a mouse or other pointing device, and they can also be manipulated by giving commands in English to a “card agent” implicit in the Card World. A pile of cards consists of cards whose images overlap. When a pile of cards has been stacked by the card agent, the images of the cards are displayed on top of each other, slightly offset, so that only the upper face of the top card in the pile is visible, but a side and end of the pile are visible too. Initially CardWorld shows one stack of cards – a complete deck of 52 standard cards plus two jokers. The card images used are from the web site http://www.jfitz.com/cards/ .

Note: Pointing gestures to cards, piles, and places on the table must be made before the Enter key is pressed at the end of a command in English. That is, pointing can be done before or after a command is typed, provided it is done before the Enter key is pressed to begin interpretation of the command.

Direct manipulations:

Commands in English:

Semantics

  1. by clicking on them with a mouse (or other pointing device) and using deictic determiners or pronouns in the noun phrase of the command: “this”, “that”, “these”, “those”, “it”, “them”, or the definite determiner “the”;

  2. by using anaphoric determiners or pronouns – the same as the deictic ones or definite ones but without pointing – to refer to a card or pile(s) most recently referred to;

  3. by using quantified definite reference in various ways:

Notice that it is a non-trivial task to resolve references to objects, because pointing to a card may involve reference to the card or to its pile, and because direct manipulation permits piles to be changed between input sentences. Also, it is not trivial to resolve references to places, because pointing to a card may also involve pointing to the place where it lies.


Syntax

The syntax of commands understood by the card agent is specified by the ASD grammar Cardgram1.grm . A command may consist of one clause or of two clauses conjoined by “and”. The vocabulary of words recognized by the card agent is:

all, and, card, cards, down, each, every, face, here, it, next, now, of, one, out, over, pile, piles, separately, shuffle, spread, stack, stacks, that, the, them, then, there, these, this, those, together, turn, up.

Examples of valid commands are

    shuffle the cards

    spread them out face down

    turn this pile over and stack it here [with pointing to a pile and a place]

    turn each of the piles face up

    shuffle each pile and stack it separately face down

    shuffle the piles and stack them together here [with pointing to a place]

    turn every card over and spread them all out together

    turn over this card [with pointing to a card]

    spread out this pile [with pointing to a card in a pile]

    turn down each of the piles

    shuffle all the cards and spread them out face up here [with pointing to a place]

    [move a card away from the pile] stack the pile here


Implementation notes:

The implementation consists mainly of the following Java classes

Due to subtleties in the semantics of English, the logic for finding the referent(s) of a noun phrase is the most complicated part of the implementation. That logic is expressed mainly in the instance functions with names beginning findReferents... in the class CardAgent1. Those functions implement a decision network which selects one of the following sixteen different referent types depending on the values of the twenty different syntactic, semantic, and pragmatic decision variables enumerated below.


Object referent types:

[The abbreviations Dx, Qx and Ax are used in comments to label appropriate parts of the source code in CardWorld1 (class CardAgent1, the functions findReferents...).  They are also used in appropriate parts of the source code for CardWorld1a (class CardAgent1, function findReferents).  Finally, they are used to label final nodes in the decision networks CardDecisionNet1a.grm and CardDecisionNet1b.grm.]

Deictic (i.e., with pointing):

D1. the card most recently pointed to or moved (e.g. "that card")

D2. the pile containing the card most recently attended to (e.g. "that pile")

D3. each card in the pile containing the card most recently attended to

(e.g. "each of those cards")

D4. all of the cards (together) in the pile containing the card most

recently attended to (e.g. "all those cards", "them"?)

Quantified, universally or uniquely (i.e., all instances of or unique instances of a type):

Q1. each card on the table separately (e.g. "each card", "each of the cards", "every card")

Q2. all of the cards on the table together (e.g. "all [together]", "[all] [of] the cards")

Q3. each pile on the table separately (e.g. "each [pile] [separately]", "every pile",

"each of the piles", "all [of] the piles [separately]")

Q4. all of the piles on the table together (e.g. "all [of] the piles together"

Q5. the only card that the command would sensibly refer to

(e.g. "the card" when there is only one card by itself, the others being in multi-card piles)

Q6. the only pile that the command would sensibly refer to

(e.g. "shuffle it" or "turn the pile over" when there is only one pile containing more than one card, or "turn it over" when there is only one pile on the table, or "turn them over" when all of the cards on the table are in a single pile)

Anaphoric (i.e., backward reference without further pointing):

A1. the last card referred to (e.g. "it", or "that [card]" without pointing)

A2. the last pile referred to (e.g. "it", or "that [pile]" without pointing)

A3. the last cards referred to, separately ("them", "those [cards] [separately]")

A4. the last piles referred to, separately ("them", "those [piles] [separately]")

A5. the last cards referred to, together ("them", "those [cards] [together]")

A6. the last piles referred to, together ("them", "those [piles] [together]")

Object referent decision variables:

?1. Is the current clause the first in its sentence?

?2. Has a card been pointed to since the last sentence?

?3. Has the explicit type "card" been used in the current clause?

?4. Has the explicit type "pile" been used in the current clause?

?5. Was the previous type unexpressed, "card", or "pile"?

?6. Is the noun or pronoun in the current clause singular or plural?

?7. Is the quantity of referents expressed in the current clause one or more than one?

?8. Is a deictic pronoun or determiner ("this", "that", "these", or "those") used in the current clause? [Note: Those words may also be used anaphorically, or even as unique quantifiers.]

?9. Is an anaphoric pronoun ("it", "them") used in the current clause? [Note: Those words may also be used deictically, or even as unique quantifiers.]

?10. Has "each", "every", or "separately" been used in the current clause?

?11. Has the adverb "together" been used in the current clause?

?12. Has separate aggregation (originally expressed by "each", "every", or "separately") continued to be used in the most recent preceding clause?

?13. Can the verb in the current command apply to single cards or only to collections of two or more cards?

?14. Was there a single last card referred to before this clause?

?15. Was there a single last pile referred to before this clause?

?16. Were there two or more piles referred to before this clause?

?17. Was the last operation performed on all CARDS on the table (as opposed to all of the piles, or just some of the cards)?

?18. Is there only one pile on the table?

?19. Is there only one multi-card pile on the table?

?20. Is there only one singleton card on the table?

Further notes about CardWorld


See FurtherNotesAboutCardWorld for more details.


References:

http://en.wikipedia.org/wiki/Anaphora_%28linguistics%29

http://en.wikipedia.org/wiki/Deixis

http://en.wikipedia.org/wiki/Indexicality


last updated 2012 Sep 13