About the exercise:The text said "using Dirichlet, Categorical and Normal" but doesn't specify which variable has one distribution, which the other. Is it the same of the exercise done at lesson? Or is different?
Thank you
Let me apologise if my questions are trivial/bad posed but i'm new to this things.
Hi,
I'm adding on this so that I don't create a new thread for the same exercise: even assuming that the variables belong to the same family of distributions of the similar example in the graphical models notebook, I haven't fully understood how to make the z variables dependent on the previous one in pyro. Not specifying the underline distribution, I wanted to know if something like:
with pyro.plate('var', N):
z = pyro.sample('z', dist.some_distributions(probs=theta[theta_probs]))
theta_probs = z
is actually ok for our task. From what I've understood "with pyro.plate" treats the samples made within its context conditionally independent, but in this case I would be changing the distribution at each iteration of the loop, thus defining a dependence. Is this a proper way to encode the concept presented in the given graphical model? I have read through the examples on the pyro webpage but I am still insecure of how to properly use pyro tools.
Best regards,
Francesco
Hi Francesco,
the correctness of your answer depends on how you defined `theta`.
A colleague of yours asked me a similar question about the conditional dependence between the z variables, so I think I should give you all a hint.
If you put a Dirichlet distribution on theta_k, then you can sample z_t from a categorical distribution, where the chosen theta depends on the previous sampled z_{t-1}. Notice that this does not hold just for the first z in the sequence, since there in no connection to previous z nodes. As an additional clue, I attach a more explicative representation of the same graph.
Ginevra
Hi Ginevra,
Thank you for the answer. I feel like I got your hint and I have defined theta accordingly. I am still missing a couple of things though:
1) "Notice that this does not hold just for the first z in the sequence, since there in no connection to previous z nodes": should we draw it randomly with a uniform probability? That's my best guess as for now.
2) I still don't understand how to properly use pyro in this context. Zs are not conditionally independent with respect to their precedent so it would seem to me that using "with pyro.plate" would be a mistake, since it makes the computations in the context conditionally independent; this is what you've explained us in the notebook. This is still a bit obscure to me: pyro's documentation talks about "batch dimensions being independent" (http://pyro.ai/examples/tensor_shapes.html) and honestly I don't get the meaning of this. I guess it is working as a sort of wrapper of a for loop, allowing us to repeat a certain calculation and get the tensor of samples as a result, somehow guaranteeing that these samples are conditionally independent one another. Would I achieve a more proper result (i.e. properly conditioned on the previous z) by using a standard for loop? Does my code snippet above make any sense (i.e. changing the distribution at each iteration while asking for conditional independence) or is it contradictory?
Sorry for being overly pedantic.
Best regards,
Francesco Brand
Hi Francesco,
keep in mind that this exercise does not admit a unique possible solution and that my answer refers to the generative process I thought about when building the exercise. You may come up with a different plausible answer.
1) You could simply use the same distribution for all the z_t, in such a way that z_1 only depends on theta, while the others also depend on z_{t-1}.
2) You're right, the z_t are not independent, so you are not supposed to use pyro.plate on them. In fact, if you look at the original graph, only theta and mu vectors are represented in plate notation. You can use both a for loop or sequential sampling calls, since there are only three variables in this case.
Ginevra