~ Evaluating results ~
Back to advanced searching lore   [Introduction]   

[Essays]   

[Sielaff's lessons]   

[Old methods, recent syntoms]   

[Negative evaluation rules]   


Introduction

You may have seen, in the basic part of searchlores, how important it is to apply (and acquire) a systematic approach in order to evaluate the tools you will use for searching, and understand what they will allow you to fetch - or keep you from fetching - before even starting your queries.
Well, that's nothing. It is actually even more important for you to acquire and apply a systematic approach when evaluating the (many) documents, clues and results that you will gather as a result of your (hopefully well-prepared) queries.

See, the relation between the small tiny signal you are searching for on the deep web and the broad loud noise you will always fetch "around it" makes it very important that you learn how to filter... how to "strain" your results. And you will have to do this quickly and competently, even when searching matters and fields in which you are no expert. Luckily there are some general rules that you can follow.

Since I had the honour to have had myself an incredible expert in this field - as my teacher - for many years, I'll use the following simple 'lore' approach to underline the importance of a correct evaluation when searching...



Sielaff's lessons

Old techniques can be very useful for modern evaluation purposes
by fravia+, first published in March 2000, Last updated in January 2004.
...Yep, I remember Professor Frjthiof Sielaff's lessons about scripts' evaluation lores... let me state that those old lessons are VERY IMPORTANT for anyone that searches the web nowadays... no, I will not "state" anything (it would contradict the very purpose of this very teaching)... just read the following "lore" and judge by yourself...



We sat there waiting and ziemlich gespannt, January, quite cold in Berlin, but, after all, that was the reason we were there: Sielaff was still teaching, against all political odds.
And he came, with his classical quarter of an hour "academic delay"... He came "herein" leaning on one side, because he carried at least a dozen books under his left arm, and then he sat down, without even looking at us, and let all his books fall on the teacher's desk.
Mind you, this was the most cherished post-university course for historians of the early middle ages in Europe in those days, so you can imagine how silent and attentive we all (students from all over Europe) were...
He still did not look at us. "These books" - he began - "deal all, more or less, with the same subject: a history of Danemark in the middle ages. Now please try to supersede the subject: actually the subject could be completely different, it would not matter in the least per se: the point is that you should bei Gelegenheit learn how to EVALUATE all kind of books BEFORE buying and/or reading them (to do it afterwards is not easy either, by the way, but at least that kind of evaluation requires more obvious skills)."
Sielaff looked sharply at his audience. "I imagine" - he said - "that you already know that most of the books... and data... around us are next to useless, don't you?"
Dunnow about the others, but this took me by surprise: I had always thought, na´vely enough, that anything that was published must have had some sort of "value".
"This book..." - Sielaff began, taking the first book out of his pile - "...is titled, quite appropriately for our today example: 'History of Danemark in the Middle ages'..." - he paused and looked at us - "...unfortunately, the Author, as stated by himself in the introduction, does not know danish at all, therefore... " - he suddendly threw the book on a far corner of the teaching desk in disgust.

YOU CANNOT WRITE ABOUT SOMETHING WHICH IS LANGUAGE-RELATED IF YOU DO NOT KNOW THAT LANGUAGE
Obvious but very often underestimated

"This book, on the other hand..." - he continued, picking another book from his pile - "... which is titled 'Danemark between 500 and 1200' has been written by an Author that actually does happen to know danish... unfortunately (for him, on the other side quite fortunately for those among us that want to appreciate books like this on the fly) the same Author wrote - before this book - a book titled 'Cactusses and other desert's plants' and, should that not be enough for you, he wrote - a short time after having published the book I am helding right now in my hand - another book titled 'Aquarium techniques for home and profit'.
The book flow flattering on the far corner on the desk, where it hit the previous one.

YOU CANNOT WRITE ABOUT UNRELATED THINGS AND PRETEND YOU REALLY KNOW SOMETHING ABOUT ANYONE OF THEM
Obvious but very often underestimated

"I am sure you begin to understand, now..." - he said - "...but let's continue, because there is MUCH more to understand and reverse... here is another book: 'Danemark in the middle ages' - and he fished a third book from his pile - "that has been written by someone that definitely knows danish, that only worked on danish history of the middle ages, and that happens to be a recognized authority in such matters" - he paused and then throw the book disgusted on the "crap" side of his desk - "I know you wont like what I will now say" - he added - "but it is quite important nevertheless: the Author wrote this book when he was only THIRTY years old!" - in fact we were all still approaching thirty ourselves and therefore more or less speechless - "I fear you'll probably don't dig it right now, but believe me: if you want to be really sure someone knows anything about what he's writing about - especially in complex sciences like early medieval history -, you wait until he is AT LEAST fifty years old, and even in that case you should by all means take GREAT care: most of the so-called "experts" are oft enough just releasing hot air bubbles. This has nothing to do with danish history specifically, of course... it happens anywhere, anyhow, anyabout..."


YOU CANNOT TRUST ANYONE THAT DID NOT SPEND A RELEVANT AMOUNT OF TIME ON A SINGLE AND SPECIFIC THEME
Obvious but very often underestimated

"You see..." - continued Sielaff - "...the fact that the previous author worked a lot on a single subject may be relevant for the books he wrote / will have written at the END of his career. But this does not guarantee nothing at all about the books he may have written at the beginning. And now we come to this article: 'About mediaeval danemark' which was written by an expert on danish medieval history, who was fifty himself at the time of writing. The article appeared in november 1982 on a university monthly collection..." - silence followed, we were trying to guess - "...ahem, I repeat: it appeared in november on a monthly..." - more silence: we didn't know what to say - "...ok: if you don't know it yet, then you better learn it right now: monthly publications are tricky - especially university ones.    They actually HAVE to publish their 12 issues year after year in order to survive... and get public money for the following years... yet the quality varies considerably, and though it may be relatively easy to find some sound and interesting material for the first, say, five or six months, you'll have to scratch the bottom of the 'Topf' to be able to fill and publish all 12 issues... therefore, my dear students, everything published 'from september onwards' should be regarderd with suspect... mind you: it does not MEAN that the stuff is crap, it is only LESS PROBABLE that you'll have some outstanding work there..."

YOU SHOULD ALWAYS TAKE INTO ACCOUNT THE ECONOMIC RATIOS BEHIND ANY PUBLISHING EFFORT
You should always take into account economic ratios behind anything, btw...

"And now we are approaching the real 'kern' of today's session." - Sielaff yawned - "Here you have another book: History of Danemark through the middle ages." - He raised a hand holding a thick book from his pile. - "This was written by a recognized expert of danish history, towards the end of his life, building on many essays he wrote before on the same argument. It was published by the most important danish editor, translated in German, English and French."
- A small pause - "It has no footnotes, only endnotes."
Sielaff coughed, smiling sadly. - "This basically means that you should 'believe' the historical reconstruction of the Author, instead of having the possibility - and the ease - to confute 'at every step' his writings."
"So even the 'formatting' of your target resource is important as well, and please note that - most of the time - said formatting will NOT be casual."
He threw the book away in disgust.

FORM MATTERS ALMOST AS MUCH AS SUBSTANCE, MOREOVER YOU SHOULD ALWAYS CONSIDER HOW 'OPEN TO QUESTIONING' YOUR TARGETS PRESENT THEMSELVES
Some Authors only refer to secondary sources, if ever. Others will bend backwards to avoid giving readers direct access to a primary source that could be interpreted differently.


"Today's 'proseminar' is almost finished."
"Guess you may enjoy this book." - Sielaff held in his hands the last book of the pile.
"Titled "A short history of Danemark in the middle ages" this book was written by a recognized expert of the matter, 60 years old at the time of writing, has footnotes, uses correctly the sources... as far as I can judge... seems 'in ordnung', has all the characteristics of a sound book..." - Sielaff opened the book and sniffed among the pages. - "It even smells good."
Sielaff's head jerked back. - "Yet you should NOT trust its assertions, not in the least: Check the sources yourselves, confute the 'truths' and the 'discoveries' of this Author... think at the possible alternative interpretations..." - Sielaff hands waved in the air. - "It's only WORDS! Never forget it: books, knowledge... just a cobweb of words, behind them, often enough: nothing.
A cobweb of theories, that's "science". It's up to you to dispel it.
Sielaff's eyes pierced the audience. - "A critical mind, that's your only weapon inside the dark forests of bogus knowledge you will have to cross again and again, your critical mind... never ever allow it to get dull."
Sielaff put the book down, closed it gently, and left.

NEVER EVER TRUST YOUR SOURCES ONLY BECAUSE THEY KINDA "APPEAR" TRUSTWORTHTY
A critical mind, that's your only weapon inside the forests of bogus knowledge, never ever allow it to get dull.



Petit image

(c) III Millennium: [fravia+], all rights reserved, coupla wrongs reversed