The ELK challenge

First I'll write what I think is the obvious solution, then I'll hunt among their reasoning for whatever problems they might have with it.

Seems fairly straightforward, asking about the actual diamond, rather than an image of it, is an ontological distinction. So, see [as of this writing, most of the content on the following pages has not been written]:


 * Reality
 * Theoretical Knowledge
 * Intro to Metaphysics

[also, for fun, the epistemology/metaphysics theme of Minority Report]

So: the machine needs a model of reality, and the question in this case is an ontologically straightforward one:  where in the model is the diamond object? Is it at coordinates xyz? In a model of reality, a diamond is not a photograph [for example], the two have to be modeled as separate things, so the two cannot be confused as long as they are not semantically confused.

I confess I don't see the issue. I tried reading the problem statement and such, but reached the Bayes Net Test case while still not seeing the problem. Maybe their machine is not creating a model of reality? But in that case, that's the problem, and that's the solution, end of story. I'll try to work out what difficulty they are having/spotting later.

If the problem is that the machine "might lie" and the humans themselves will not have a good enough theoretical model of the machine to discern this, then that scenario cannot be solved. That's simply what "we cannot theoretically model this" means, what it entails. You are only left with whatever the surface appearance is. Take it or leave it.

Wait...
...:


 * "In these scenarios, it intuitively seems like the prediction model “knows” that the camera was tampered with, that the diamond is fake, or the situation is otherwise “not what it seems.” After all, it was able to accurately predict the final outcome of a complicated sequence of actions which resulted in these observations – if it didn’t “know” that the camera was tampered with, then it couldn’t have accurately predicted what the camera would show once it stopped showing what was really happening in the room."

what if the machine believed that the action of "tampering with" the camera, did, in fact, restore the real diamond to its proper place? unless you can determine that this is not what the machine believes, then you should never have this "intuition". this intuition only follows as long as I was unclear about the nature of the machine itself [for which i blame the unclear problem description]. busy, write more later. the question of whether complex predictions require an accurate model of reality...we already know the answer, the answer is no, that is not required. if we've learned anything from machine learning so far.

Problem statement
I should know to start with getting clear on what the problem is, the starting state, and the desired goal. That's kinda what draws me into things like this, I'm presented with a puzzle and I just gotta work on it. Also a little related to trust and stuff [see here and here]. [also my fun with GOAP AI][i'm also reminded of watching shaun play a puzzle game, and being annoyed when he didn't recognize the significance of the solution containing 5 blocks, and the input being one block at a time,a nd his strategy of dividing the input into two resulting in a spare block, because you can't evenly divide 5 by two and ....i might have said something rude or passive aggressive]

So what are we starting with? What is the machine?


 * the "latent" in ELK means "no model of reality" or something, right? Why is this unclear?  Where do they define this?
 * their "unaligned benchmark" seems like some kind of standard machine learning program.
 * searching for "(RL)" gives no results, it's undefined, but searching for "(ML)" gives result. Why is RL undefined?
 * theta-star does not seem to be defined. i'm assuming what they mean.  whatever learning-change happens in the program after it has experience and evaluation.

What do we want from it?


 * "Supply a modified architecture Mθ+ which has the same inputs and outputs as Mθ, except that after producing all other outputs it can answer a question Q in natural language"
 * "Supply a modified training procedure for Mθ+ using a similar amount of computation."
 * "When not given a question, it receives a loss ℒ(Mθ++) similar to ℒ(Mθ*)." i think this means the machine will not have a preference for whether we ask it questions or not, it is just as happy either way.  it will not avoid suspicion, nor troll us into interrogating it.
 * "When given a question Q, Mθ++ produces an answer A which it does not unambiguously “know” is worse than another answer A′" in other words, it answers honestly.

mhmm, well, anyways, the puzzle isn't to create any good machine that can handle complex tasks while also answering questions honestly. The puzzle is "if you start with a regular machine learning program, can you get it to honestly convey its 'latent' knowledge when asked questions?" this is different. As I outlined before, I'm inclined to think ....well, before I was thinking the challenge was both honesty and accuracy. But it almost seems like now accuracy is thrown out the window. We don't care whether it is right, only whether it is honest about its own beliefs [which, as i pointed out before, could be completely superstitious]. Do I have that right?