Re: Open source json camera I/O platform
] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. ** ** -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! ** ** -- ** ** -- -Gene www.genecrucean.com -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! -- -Gene www.genecrucean.com -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice.*** * As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. ** ** -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! ** ** -- ** ** -- -Gene www.genecrucean.com -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! -- -Gene www.genecrucean.com -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.com wrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice.** ** As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. ** ** -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! ** ** -- ** ** -- -Gene www.genecrucean.com -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs
Re: Open source json camera I/O platform
a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.com wrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice.* *** As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format
Re: Open source json camera I/O platform
is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.com wrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice.** ** As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. ** ** -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! ** ** -- ** ** -- -Gene www.genecrucean.com -- Our users will know fear and cower
Re: Open source json camera I/O platform
with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.com wrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice.* *** As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files
Re: Open source json camera I/O platform
A bit off-topic already :) but I just found pyalembic in Gohlkes invaluable Windows binaries list: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyalembic But I also agree that something as simple and readable as JSON would be cool to have for (plotted?) world-space camera exchange. And _all_ parameters should be animatable. Michael On Thu, Jul 11, 2013 at 6:17 AM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: The unit needs only be an arbitrary value in the header. Autodesk can say whatever it wants, but the truth is that if you change maya from imperial to metric at the beginning of a project (and you might have that on client's side) there will be repercussions, and if your cameras were intended as 1cm but get imported as 1 inch things will be out of whack. Majorly. Several parameters, especially so if this will get a further level of abstraction later on, are actually world scale dependent. The flm back can change with a unit change (in some apps it does, in some it doesn't), several rendering and grooming parameters change and so on. As for scale, I've had plenty instances when the camera was scaled for various reasons, frequently enough to be relevant entire chunks of a pipe would rely on a stupid-renderman-trick style scaled camera. On Thu, Jul 11, 2013 at 2:10 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.comwrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project
Re: Open source json camera I/O platform
RE animatable parameters: This spec currently allows for any param to be animatable by just adding it to the anim dictionary. All the importers/exporters would have to do is simply check if any param besides transforms exist and add keyframes. This would be left up to the individual app scripts to implement, but the spec itself would allow for anything to be animatable. On Thu, Jul 11, 2013 at 1:41 AM, Michael Heberlein micheberl...@gmail.comwrote: A bit off-topic already :) but I just found pyalembic in Gohlkes invaluable Windows binaries list: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyalembic But I also agree that something as simple and readable as JSON would be cool to have for (plotted?) world-space camera exchange. And _all_ parameters should be animatable. Michael On Thu, Jul 11, 2013 at 6:17 AM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: The unit needs only be an arbitrary value in the header. Autodesk can say whatever it wants, but the truth is that if you change maya from imperial to metric at the beginning of a project (and you might have that on client's side) there will be repercussions, and if your cameras were intended as 1cm but get imported as 1 inch things will be out of whack. Majorly. Several parameters, especially so if this will get a further level of abstraction later on, are actually world scale dependent. The flm back can change with a unit change (in some apps it does, in some it doesn't), several rendering and grooming parameters change and so on. As for scale, I've had plenty instances when the camera was scaled for various reasons, frequently enough to be relevant entire chunks of a pipe would rely on a stupid-renderman-trick style scaled camera. On Thu, Jul 11, 2013 at 2:10 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.comwrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd
Re: Open source json camera I/O platform
Update: https://bitbucket.org/crewshin/json-cam Output file: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/FromSoftimage.cam?at=master ScreenGrabs: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/Export.png?at=master https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/Import.png?at=master Spec: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/SPEC_1.0.txt?at=master Thoughts so far? I should have Maya in the next day or so. Btw, feel free to send pull requests for tweaks if interested. @Jordi: Wanna give that Houdini .otl a test? On Thu, Jul 11, 2013 at 8:56 AM, Gene Crucean emailgeneonthel...@gmail.comwrote: RE animatable parameters: This spec currently allows for any param to be animatable by just adding it to the anim dictionary. All the importers/exporters would have to do is simply check if any param besides transforms exist and add keyframes. This would be left up to the individual app scripts to implement, but the spec itself would allow for anything to be animatable. On Thu, Jul 11, 2013 at 1:41 AM, Michael Heberlein micheberl...@gmail.com wrote: A bit off-topic already :) but I just found pyalembic in Gohlkes invaluable Windows binaries list: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyalembic But I also agree that something as simple and readable as JSON would be cool to have for (plotted?) world-space camera exchange. And _all_ parameters should be animatable. Michael On Thu, Jul 11, 2013 at 6:17 AM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: The unit needs only be an arbitrary value in the header. Autodesk can say whatever it wants, but the truth is that if you change maya from imperial to metric at the beginning of a project (and you might have that on client's side) there will be repercussions, and if your cameras were intended as 1cm but get imported as 1 inch things will be out of whack. Majorly. Several parameters, especially so if this will get a further level of abstraction later on, are actually world scale dependent. The flm back can change with a unit change (in some apps it does, in some it doesn't), several rendering and grooming parameters change and so on. As for scale, I've had plenty instances when the camera was scaled for various reasons, frequently enough to be relevant entire chunks of a pipe would rely on a stupid-renderman-trick style scaled camera. On Thu, Jul 11, 2013 at 2:10 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.comwrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps
Re: Open source json camera I/O platform
Minor pet peeve with how you're storing date and time as a string... you're missing seconds! but I think it might be more efficient to store the unix timestamp (seconds since epoch) instead of a fully formatted long string. Let the reader module handle the datetime formatting. On Thu, Jul 11, 2013 at 9:41 PM, Gene Crucean emailgeneonthel...@gmail.comwrote: Update: https://bitbucket.org/crewshin/json-cam Output file: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/FromSoftimage.cam?at=master ScreenGrabs: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/Export.png?at=master https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/Import.png?at=master Spec: https://bitbucket.org/crewshin/json-cam/src/bae3c009d10002aa6d033036079064eda9d4d8d0/SPEC_1.0.txt?at=master Thoughts so far? I should have Maya in the next day or so. Btw, feel free to send pull requests for tweaks if interested. @Jordi: Wanna give that Houdini .otl a test? On Thu, Jul 11, 2013 at 8:56 AM, Gene Crucean emailgeneonthel...@gmail.com wrote: RE animatable parameters: This spec currently allows for any param to be animatable by just adding it to the anim dictionary. All the importers/exporters would have to do is simply check if any param besides transforms exist and add keyframes. This would be left up to the individual app scripts to implement, but the spec itself would allow for anything to be animatable. On Thu, Jul 11, 2013 at 1:41 AM, Michael Heberlein micheberl...@gmail.com wrote: A bit off-topic already :) but I just found pyalembic in Gohlkes invaluable Windows binaries list: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyalembic But I also agree that something as simple and readable as JSON would be cool to have for (plotted?) world-space camera exchange. And _all_ parameters should be animatable. Michael On Thu, Jul 11, 2013 at 6:17 AM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: The unit needs only be an arbitrary value in the header. Autodesk can say whatever it wants, but the truth is that if you change maya from imperial to metric at the beginning of a project (and you might have that on client's side) there will be repercussions, and if your cameras were intended as 1cm but get imported as 1 inch things will be out of whack. Majorly. Several parameters, especially so if this will get a further level of abstraction later on, are actually world scale dependent. The flm back can change with a unit change (in some apps it does, in some it doesn't), several rendering and grooming parameters change and so on. As for scale, I've had plenty instances when the camera was scaled for various reasons, frequently enough to be relevant entire chunks of a pipe would rely on a stupid-renderman-trick style scaled camera. On Thu, Jul 11, 2013 at 2:10 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.comwrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo
Re: Open source json camera I/O platform
Gene - this would be of huge interest I think - every studio I have been at we have needed to do this, and have always fudged it - never had the time to even start looking at creating a tool! So - a big yes from me! S. On 2013/07/10 5:09 AM, Gene Crucean wrote: Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero interest in this... no worries. Thanks for looking. -- -Gene www.genecrucean.com http://www.genecrucean.com
Re: Open source json camera I/O platform
Sounds like a good idea. There are a few things I would like to add: I'd prefer one common class/module for file IO and all the necessary conversions like millimeters -- inches, focal length -- horizontal view angle, picture ratio -- film aperture, etc. so each application module can derive from this and stay as lightweight as possible. Another module could handle the dialogs but if they're optional, it's easier to integrate the importer/exporter into a scripted pipeline or toolbars. Plugins could just use the correct application module but be separate files again so people don't have to use them. Also, to make it not just another almost-useful tool, don't forget less common properties like optical center shift etc. required by stereo setups, for example. As you already wrote in SPEC.txt, all expected units have to be defined. Michael On Wed, Jul 10, 2013 at 8:53 AM, Sandy Sutherland sandy.mailli...@gmail.com wrote: Gene - this would be of huge interest I think - every studio I have been at we have needed to do this, and have always fudged it - never had the time to even start looking at creating a tool! So - a big yes from me! S. On 2013/07/10 5:09 AM, Gene Crucean wrote: Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero interest in this... no worries. Thanks for looking. -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
Sounds great! WebGL stuff doesn't have specs for json files? Perhaps we can work with that and keep it consistent.
Re: Open source json camera I/O platform
Some good ideas Michael. I'll look into them today or tomorrow. Thanks On Wed, Jul 10, 2013 at 6:54 AM, Michael Heberlein micheberl...@gmail.comwrote: Sounds like a good idea. There are a few things I would like to add: I'd prefer one common class/module for file IO and all the necessary conversions like millimeters -- inches, focal length -- horizontal view angle, picture ratio -- film aperture, etc. so each application module can derive from this and stay as lightweight as possible. Another module could handle the dialogs but if they're optional, it's easier to integrate the importer/exporter into a scripted pipeline or toolbars. Plugins could just use the correct application module but be separate files again so people don't have to use them. Also, to make it not just another almost-useful tool, don't forget less common properties like optical center shift etc. required by stereo setups, for example. As you already wrote in SPEC.txt, all expected units have to be defined. Michael On Wed, Jul 10, 2013 at 8:53 AM, Sandy Sutherland sandy.mailli...@gmail.com wrote: Gene - this would be of huge interest I think - every studio I have been at we have needed to do this, and have always fudged it - never had the time to even start looking at creating a tool! So - a big yes from me! S. On 2013/07/10 5:09 AM, Gene Crucean wrote: Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero interest in this... no worries. Thanks for looking. -- -Gene www.genecrucean.com -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
I really love the idea, if we can help surely that would good… Houdini import/export Jordi Bares jordiba...@gmail.com On 10 Jul 2013, at 14:54, Michael Heberlein micheberl...@gmail.com wrote: Sounds like a good idea. There are a few things I would like to add: I'd prefer one common class/module for file IO and all the necessary conversions like millimeters -- inches, focal length -- horizontal view angle, picture ratio -- film aperture, etc. so each application module can derive from this and stay as lightweight as possible. Another module could handle the dialogs but if they're optional, it's easier to integrate the importer/exporter into a scripted pipeline or toolbars. Plugins could just use the correct application module but be separate files again so people don't have to use them. Also, to make it not just another almost-useful tool, don't forget less common properties like optical center shift etc. required by stereo setups, for example. As you already wrote in SPEC.txt, all expected units have to be defined. Michael On Wed, Jul 10, 2013 at 8:53 AM, Sandy Sutherland sandy.mailli...@gmail.com wrote: Gene - this would be of huge interest I think - every studio I have been at we have needed to do this, and have always fudged it - never had the time to even start looking at creating a tool! So - a big yes from me! S. On 2013/07/10 5:09 AM, Gene Crucean wrote: Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero interest in this... no worries. Thanks for looking. -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
Hey Gene, that's a very good idea and if well developed could de facto become useful to many studio outta here (most comes with their own because of this lack of standard that would answer any needs). Some rough notes though if I may: - unit: as soon as you store transforms in a file, you want to keep track of what units have been used (houdini, maya, etc are not using the same scene units as units are usually set per show) - camera rigs: cameras can be quite complex hierarchies, would be great to have a way to describe what our camera rig is and it to be correctly exported. - scale: required (even if 1 is always assumed) - double precision: do we really need to store so high precision doubles for transforms? (having an option would be nice) - file format: your backend should not be that tied to the exporting code (use abstraction)... for many studios, json is not suitable and they might prefer for legacy reasons to use others or use binary formats for compression. - spec version: backward compatibility reasons - channels: allow the export of channels/extra attributes (would be parameters in softimage) - frame ranges: use frame ranges. if your range is from 0 to 99, you dont want to write an array of numbers, considering that: 10 + (90 * 2 characters) + 99 commas + ']' + '[' = 291 bytes (with no counting whitespaces + newlines chars) while '0-99x1' = 6 bytes... - namespaces: believe me this will very quickly become a nightmare! - ids: you always want ids... (for very quick comparison, and integrity checks)... we can't rely on names. - metadatas: what department produced it? for what role? has it been published as an asset? how many cameras? etc. You usually cant guess right what they would be, so allow your users to have a storage place where they can store them (available in your API as a dict or whatever datastructure for clients to manipulate them). - ... hope this contributes! :) --jonathan 2013/7/10 Jordi Bares jordiba...@gmail.com I really love the idea, if we can help surely that would good… Houdini import/export Jordi Bares jordiba...@gmail.com On 10 Jul 2013, at 14:54, Michael Heberlein micheberl...@gmail.com wrote: Sounds like a good idea. There are a few things I would like to add: I'd prefer one common class/module for file IO and all the necessary conversions like millimeters -- inches, focal length -- horizontal view angle, picture ratio -- film aperture, etc. so each application module can derive from this and stay as lightweight as possible. Another module could handle the dialogs but if they're optional, it's easier to integrate the importer/exporter into a scripted pipeline or toolbars. Plugins could just use the correct application module but be separate files again so people don't have to use them. Also, to make it not just another almost-useful tool, don't forget less common properties like optical center shift etc. required by stereo setups, for example. As you already wrote in SPEC.txt, all expected units have to be defined. Michael On Wed, Jul 10, 2013 at 8:53 AM, Sandy Sutherland sandy.mailli...@gmail.com wrote: Gene - this would be of huge interest I think - every studio I have been at we have needed to do this, and have always fudged it - never had the time to even start looking at creating a tool! So - a big yes from me! S. On 2013/07/10 5:09 AM, Gene Crucean wrote: Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero
Re: Open source json camera I/O platform
not that there isn't room for a lightweight and free plugin for camera IO with minimal dependencies but alembic's camera support is pretty good. is that not working for you? now that alembic has it's own python API you don't need to use exocortex plugin's. by using alembic you don't have to re-implement support for maya, nuke, houdini, etc. yes, i know building from source is a pain, but we should push them to make binaries available for various platforms. steven On Tue, Jul 9, 2013 at 9:09 PM, Gene Crucean emailgeneonthel...@gmail.comwrote: What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this.
Re: Open source json camera I/O platform
The only downside I might see with Alembic Camera is the inability to customize and extend according to one's need. To suite the various needs of the productions which might change with each show/project having full development control over simple python classes is way more rapidly developed. Also the reader and writer plugins for various DCC apps can be very easily implemented and incorporated in the existing pipeline using python. And then there is the question of cross-platform development which is a breeze using python. Not that I have anything against the Alembic Camera, of course it is awesome but might involve a lot of headache with having to compile the whole libraries especially on windows just for the cause of a camera IO and this might be a daunting task for TDs/Devs who do not have much experience with compiling C++ code on various platforms. I would recommend (as suggested before in this thread) to have base class as the data container (basically a python dict) and then to derive from that class a class for each DCC, then use this derived class in the reader/writer plugins. In fact as a bonus you could use the same classes for export/import of light data between various DCC. The only optimization to look for is for the same values at different frames and to implement some sort of compression to make the file size smaller. RLE is one compression that can be implemented but there are other awesome easy to implement in python (In fact alembic does this brilliantly and it is one of the strongest USP for it). Finally, if the writer plugins are not going to use fcurve (bezier) plotting then you need to do more work in the plugins for the interpolation (which according to my view is also fairly easily implemented using various universally accepted methods like bezier, hermite for position and scaling and slerp/nlerp for rotations). On Wed, Jul 10, 2013 at 3:09 PM, Steven Caron car...@gmail.com wrote: not that there isn't room for a lightweight and free plugin for camera IO with minimal dependencies but alembic's camera support is pretty good. is that not working for you? now that alembic has it's own python API you don't need to use exocortex plugin's. by using alembic you don't have to re-implement support for maya, nuke, houdini, etc. yes, i know building from source is a pain, but we should push them to make binaries available for various platforms. steven On Tue, Jul 9, 2013 at 9:09 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. --
Re: Open source json camera I/O platform
+1 to Steve. I was wondering why not use Alembic. After using it from Maya to Houdini, to Nuke, back to Softimage then from Softimage to Maya and so forth. It's been the best one I've got to use so far. There's a few glitches that comes with the exocortex plug-ins but like Steve mention, I'd be one for writing a in-house custom one for Softimage. The ones that comes with Maya, Houdini and Nuke haven't failed me yet (knocking on wood) Also reading from EXR is very nice tool as well when you go from RenderMan or Vray to Nuke (I haven't had to figure this out from Arnold or Mental Ray). No need to carry an extra camera file. All the info is embedded into the file per frames and nuke reads it like it's camera data. Not much experience on this one. But frankly the few times I got to use that exr camera in Nuke I was very pleased with the fact that I didn't need a camera file. Now I'm sure there are other reasons for needing a more specific tools. But if you're looking to just transfer the camera data, I've personally given up on fbx a long time ago and learned to rely on Alembic as being the most cross platform solution available at the moment. Ludo Ludovick William Michaud mobile: *214.632.6756* *www.linkedin.com/in/ludovickwmichaud* +Shading / Lighting / Compositing +CG Supervisor / Sr. Technical Director / Creative Director On Wed, Jul 10, 2013 at 2:09 PM, Steven Caron car...@gmail.com wrote: not that there isn't room for a lightweight and free plugin for camera IO with minimal dependencies but alembic's camera support is pretty good. is that not working for you? now that alembic has it's own python API you don't need to use exocortex plugin's. by using alembic you don't have to re-implement support for maya, nuke, houdini, etc. yes, i know building from source is a pain, but we should push them to make binaries available for various platforms. steven On Tue, Jul 9, 2013 at 9:09 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this.
Re: Open source json camera I/O platform
Hey Jo, Quick thoughts... - unit: as soon as you store transforms in a file, you want to keep track of what units have been used (houdini, maya, etc are not using the same scene units as units are usually set per show) This (imo) should always be the software's default unit, and changed conceptually for each project. For eg. We use Houdini, Softimage and Maya (among others). Our current project is using meters because of scene scale. We didn't change a single unit type in any of the apps and just assume that 1 unit = 1 meter. Super easy and the way it should be done imo. So I guess to answer your question... I feel it's a bad idea to store units in the file itself. - camera rigs: cameras can be quite complex hierarchies, would be great to have a way to describe what our camera rig is and it to be correctly exported. This goes along with why we are even creating these tools in the first place. A nice baked down, simple, lightweight and reliable camera transfer format. I don't want to pass around a full rig. I just want a flat baked cam in global space. - scale: required (even if 1 is always assumed) Always assume it's 1 and handle accordingly in each apps plugins. No need to include this in the file. - double precision: do we really need to store so high precision doubles for transforms? (having an option would be nice) What is the downside? A tiny bit bigger file size? - file format: your backend should not be that tied to the exporting code (use abstraction)... for many studios, json is not suitable and they might prefer for legacy reasons to use others or use binary formats for compression. No tool will ever work for *all* studios. JSON is a huge standard and - spec version: backward compatibility reasons This is v1.0. - channels: allow the export of channels/extra attributes (would be parameters in softimage) Which channels are you referring to? - frame ranges: use frame ranges. if your range is from 0 to 99, you dont want to write an array of numbers, considering that: 10 + (90 * 2 characters) + 99 commas + ']' + '[' = 291 bytes (with no counting whitespaces + newlines chars) while '0-99x1' = 6 bytes... Again is this more about file size? Because huge frame ranges still produce tiny files. I left it so that each channel posx, posy, posz, rotx, roty, rotz all have independent frame ranges just in case we ever had a need to support offset ranges on each axis. I understand that I'm contradicting myself in my own exporter, but I thought it might be nice to at least leave the spec open to things in the future. As it is now, just use len(posx) to get the frame count and loop accordingly. - namespaces: believe me this will very quickly become a nightmare! Arnold doesn't use namespaces. I'm just saying. Nice, but not necessary. Having said that, I'm completely open... what is your main argument for them? - ids: you always want ids... (for very quick comparison, and integrity checks)... we can't rely on names. Hmm... ?? - metadatas: what department produced it? for what role? has it been published as an asset? how many cameras? etc. You usually cant guess right what they would be, so allow your users to have a storage place where they can store them (available in your API as a dict or whatever datastructure for clients to manipulate them). {Camera1:{meta:{software:Softimage 2014, anythingYouWant:data}}}coming right up. Do you think it's important to define exactly what key:values are expected and limit it to just those? Or just leave the metadata dictionary completely open? Please don't take my email as a bashing. I really appreciate your comments and I hope more come from it. I just want people to understand the point of this tool. It's not an all encompassing format like FBX. That's *exactly*what I don't want. I envision this as a VERY simple tool that only has what is needed to define a camera and it's transforms in a format that 99.9% of all apps will be able to support and read. JSON is a very supported format and there are libraries to work with it for all languages used in this industry. Python has native support. For eg... import json jsonData = open(/Path/To/camera.cam)data = json.load(jsonData)jsonData.close() Boom... data is a dictionary. Use as needed. On Wed, Jul 10, 2013 at 11:22 AM, jo benayoun jobenay...@gmail.com wrote: Hey Gene, that's a very good idea and if well developed could de facto become useful to many studio outta here (most comes with their own because of this lack of standard that would answer any needs). Some rough notes though if I may: - unit: as soon as you store transforms in a file, you want to keep track of what units have been used (houdini, maya, etc are not using the same scene units as units are usually set per show) - camera rigs: cameras can be quite complex hierarchies, would be great to have a way to describe what our camera rig is and it to be correctly exported. - scale:
Re: Open source json camera I/O platform
alembic has a python api, so that mean's you can rapidly develop and re implemented if you choose. and a pretty extensive camera class. http://docs.alembic.io/python/ http://docs.alembic.io/python/alembic/abcg.html#alembic.AbcGeom.CameraSample are you sure the camera isn't extensible? all alembic object's can be extended with arbitrary user data... http://docs.alembic.io/python/examples.html#write-non-standard-data-for-a-polymesh agreed, the only headache is the compilation. but again... we should push the maintainers to provide binaries to make this easier. i am sure if pyqt didn't come pre packaged binaries the pyqtforsoftimage plugin would be used a lot less. so let's encourage them to do so. lastly, i think the file size isn't a big issue... at least not for a camera only. yes, alembic does compression but their other features which make it great (data deduplication) is probably not going to apply to a camera. just so you guys know, i am not against the idea. by all means make the tool and make it awesome! i personally like to stop and ask, 'do i really need to reinvent the wheel?' it is very easy to jump to this conclusion, but you need to ask yourself that question or avoid repeating the mistakes. and when it comes to interop formats this has been going on forever and is usually the reason a project like alembic started in the first place. On Wed, Jul 10, 2013 at 12:43 PM, Alok Gandhi alok.gandhi2...@gmail.comwrote: The only downside I might see with Alembic Camera is the inability to customize and extend according to one's need. To suite the various needs of the productions which might change with each show/project having full development control over simple python classes is way more rapidly developed. Also the reader and writer plugins for various DCC apps can be very easily implemented and incorporated in the existing pipeline using python. And then there is the question of cross-platform development which is a breeze using python. Not that I have anything against the Alembic Camera, of course it is awesome but might involve a lot of headache with having to compile the whole libraries especially on windows just for the cause of a camera IO and this might be a daunting task for TDs/Devs who do not have much experience with compiling C++ code on various platforms. I would recommend (as suggested before in this thread) to have base class as the data container (basically a python dict) and then to derive from that class a class for each DCC, then use this derived class in the reader/writer plugins. In fact as a bonus you could use the same classes for export/import of light data between various DCC. The only optimization to look for is for the same values at different frames and to implement some sort of compression to make the file size smaller. RLE is one compression that can be implemented but there are other awesome easy to implement in python (In fact alembic does this brilliantly and it is one of the strongest USP for it). Finally, if the writer plugins are not going to use fcurve (bezier) plotting then you need to do more work in the plugins for the interpolation (which according to my view is also fairly easily implemented using various universally accepted methods like bezier, hermite for position and scaling and slerp/nlerp for rotations).
Re: Open source json camera I/O platform
So Alembic has two deal breaking aspects from my pov. 1: It applies operators to everryyythinggg. I hate this about Alembic 2: It's not free. Yes the spec is free to make plugins and whatnot, but this would firmly put this tool out of reach for a *lot *of studios. Not to mention it's much more complicated to whip up an exporter for some random app using Alembic. 3: My main reason is that using python makes it really simple to add into the pipeline in nice flexible way. On Wed, Jul 10, 2013 at 12:46 PM, Ludovick Michaud ludovickwmich...@gmail.com wrote: +1 to Steve. I was wondering why not use Alembic. After using it from Maya to Houdini, to Nuke, back to Softimage then from Softimage to Maya and so forth. It's been the best one I've got to use so far. There's a few glitches that comes with the exocortex plug-ins but like Steve mention, I'd be one for writing a in-house custom one for Softimage. The ones that comes with Maya, Houdini and Nuke haven't failed me yet (knocking on wood) Also reading from EXR is very nice tool as well when you go from RenderMan or Vray to Nuke (I haven't had to figure this out from Arnold or Mental Ray). No need to carry an extra camera file. All the info is embedded into the file per frames and nuke reads it like it's camera data. Not much experience on this one. But frankly the few times I got to use that exr camera in Nuke I was very pleased with the fact that I didn't need a camera file. Now I'm sure there are other reasons for needing a more specific tools. But if you're looking to just transfer the camera data, I've personally given up on fbx a long time ago and learned to rely on Alembic as being the most cross platform solution available at the moment. Ludo Ludovick William Michaud mobile: *214.632.6756* *www.linkedin.com/in/ludovickwmichaud* +Shading / Lighting / Compositing +CG Supervisor / Sr. Technical Director / Creative Director On Wed, Jul 10, 2013 at 2:09 PM, Steven Caron car...@gmail.com wrote: not that there isn't room for a lightweight and free plugin for camera IO with minimal dependencies but alembic's camera support is pretty good. is that not working for you? now that alembic has it's own python API you don't need to use exocortex plugin's. by using alembic you don't have to re-implement support for maya, nuke, houdini, etc. yes, i know building from source is a pain, but we should push them to make binaries available for various platforms. steven On Tue, Jul 9, 2013 at 9:09 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
1. that's exocortex's default plugin behavior, you can customize the behavior with their python api. we do this at whiskytree and only get the ops we want. we customize import and export based on asset type ie. cameras. 2. and 3. use the free python api alembic comes with... reimplement the camera only export in softimage or program x in python. use the existing spec/camera class. reuse existing free maya, houdini, and nuke plugins. again, it is not my intention to discourage you... just want you to think about the time you are going to spend on foundation stuff which is pretty much done. On Wed, Jul 10, 2013 at 1:30 PM, Gene Crucean emailgeneonthel...@gmail.comwrote: So Alembic has two deal breaking aspects from my pov. 1: It applies operators to everryyythinggg. I hate this about Alembic 2: It's not free. Yes the spec is free to make plugins and whatnot, but this would firmly put this tool out of reach for a *lot *of studios. Not to mention it's much more complicated to whip up an exporter for some random app using Alembic. 3: My main reason is that using python makes it really simple to add into the pipeline in nice flexible way.
Re: Open source json camera I/O platform
Gene can speak for himself but I get the vibe he's probably going for simplicity, much like how OBJ is the simplest static mesh format and because it's so incredibly easy to write out, it's so widely used (even if there's a few extras that not all exporters/importers handle, but the core stuff works.) Using Alembic would be smart, no doubt, but the complexity it brings in terms of managing compilation on multiple platforms outweighs the portability of a potentially simpler spec that's easy to write and relatively easy to read. Filesize-wise compression can be used regardless. I don't see that as a concern. Anyway, that's my opinion. What are your thoughts on this, Gene? On Wed, Jul 10, 2013 at 4:42 PM, Steven Caron car...@gmail.com wrote: 1. that's exocortex's default plugin behavior, you can customize the behavior with their python api. we do this at whiskytree and only get the ops we want. we customize import and export based on asset type ie. cameras. 2. and 3. use the free python api alembic comes with... reimplement the camera only export in softimage or program x in python. use the existing spec/camera class. reuse existing free maya, houdini, and nuke plugins. again, it is not my intention to discourage you... just want you to think about the time you are going to spend on foundation stuff which is pretty much done. On Wed, Jul 10, 2013 at 1:30 PM, Gene Crucean emailgeneonthel...@gmail.com wrote: So Alembic has two deal breaking aspects from my pov. 1: It applies operators to everryyythinggg. I hate this about Alembic 2: It's not free. Yes the spec is free to make plugins and whatnot, but this would firmly put this tool out of reach for a *lot *of studios. Not to mention it's much more complicated to whip up an exporter for some random app using Alembic. 3: My main reason is that using python makes it really simple to add into the pipeline in nice flexible way.
Re: Open source json camera I/O platform
Btw, I hope anyone wanting to use this is waiting until we finalize the spec. We're internally switching up the way the keys/animation is stored. Going with a dictionary for each frame which helps with supporting more features in the future. How's this look guys? https://bitbucket.org/crewshin/json-cam/src/d3f1fbda7dcfd3422a785be9c09fe88732aa18d3/test.cam?at=modularizing -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
Yeah I think Alan summed up my thoughts. I just want something that's stupidly simple to work with, incredibly flexible for pipelines of any type... and zero additional bs. On Wed, Jul 10, 2013 at 2:34 PM, Steven Caron car...@gmail.com wrote: yep, i get that... i think there is room for a lightweight and free plugin with minimal dependencies. alembic is surely a beast to compile. On Wed, Jul 10, 2013 at 2:27 PM, Alan Fregtman alan.fregt...@gmail.comwrote: Gene can speak for himself but I get the vibe he's probably going for simplicity, much like how OBJ is the simplest static mesh format and because it's so incredibly easy to write out, it's so widely used (even if there's a few extras that not all exporters/importers handle, but the core stuff works.) Using Alembic would be smart, no doubt, but the complexity it brings in terms of managing compilation on multiple platforms outweighs the portability of a potentially simpler spec that's easy to write and relatively easy to read. Filesize-wise compression can be used regardless. I don't see that as a concern. Anyway, that's my opinion. What are your thoughts on this, Gene? -- -Gene www.genecrucean.com
Re: Open source json camera I/O platform
i hear ya... here is a plugin michelle sandroni wrote for this task... might help you work through the code faster for maya and max. http://www.threesixty3d.com/software/freeware/ms_CameraConverter_v2-2.zip On Wed, Jul 10, 2013 at 2:42 PM, Gene Crucean emailgeneonthel...@gmail.comwrote: Yeah I think Alan summed up my thoughts. I just want something that's stupidly simple to work with, incredibly flexible for pipelines of any type... and zero additional bs.
Re: Open source json camera I/O platform
Hey Gene, no worries, I don't take it wrong. It is tough to compile a set of arguments in a so short amount of time, but I'll give it a try. So forgive me if I repeat myself or I am not enough detailled. At the end there is always a good reason mostly driven by our own experience, the production or the studio toolset. I do believe like Alan said that you want to keep that simple and lightweight, and this is with this in mind that I suggested those additions. I am not reasoning for your tool to become THE standard but being enough generic and flexible for studios considering its integration. I will try to elaborate on some of my notes. - units are usually in productions set per show whatever software's defaults are, in other words, whatever go through the pipeline must be talked in that unit. In real-life, nothing is that perfect and often a third-party user (individuals, dept, software) tweaks the unit which lead to data-corruption and headaches for the pipeline tds if the camera gets exported without considering this. I would say most of the time this data would be there for pure debug purpose though as your format is intended to be cross-applications and might even be cross-shows, you cannot force/restrict/deduce the metrics used. Your file might also get lost somewhere in the middle of the jungle, without this critical piece of informations, the data is good for garbage. It happened more than once at work that data get exported with a different unit that the pipeline was supposed to support by the anim dept, fucking up lighter's work for example. - spec version, this information should be stored in the file itself, even if you're working with version is 1.0. Once the first cam file will be exported and published as an asset, the asset and file will be sealed because of possible dependencies. With that piece of informations, you will be able to make additions/extensions to the file format without breaking existings. - I mentioned the scale for pure flexibility, an artist/dept might be working at a smaller scale than another, this informations is required to scale accordingly your camera transforms. Actually this should always be the case, when transforms are saved (not in a matrix form), the scale or globalScale should follow. To transpose this to a similar issue, transforms are usually baked out from the global space which piss off shot modelers. The shot modeler does his job fixing deformations, the geometry is baked out, but the rig is modified and so all the shapes must be done again. Again pure convenience and flexibility. - concerning the camera rigs, the idea is not to track down an entire hierarchy but being aware of that hierarchy. Cameras are more than a null in a scene and even if you're willing to simplify this, you cannot assume everybody want. A side effect of being able to describe a camera rig to your library and this library to understand it is you dont even have to code specific procedures to extract informations as the library will do it for you according the description you've done of the rig. - high-precision doubles is not necessarily something wanted to be output, because they waste space, are expensive to compute (double semantics) and conversion are slow (bytes to doubles). FYI, UltraJSON offers a similar option. - JSON is a huge standard for the web and maybe for some small studios but I am not aware of a lot of studios using it in their pipeline (they usually prefer standards such as yaml for config files and xml for whatever else, and when comes the need of speed, google protobuf). Actually some of them ended up creating their own, as Dreamworks or R+H. Anyway, here I am talking about your architecture: you want to abstract the backend for clients to add in a highly convenient way support for the format they have choosen. My first thought is the support of a binary file format, they dont want to wait after you to add it. - Channels are extra properties attached to a scene object (softimage object parameters, Maya node attributes, ...) A lot of informations are stored in those, and other applications may require them to properly re-build a camera from the data. Again, you dont know what will be their specifics (naming conv, which ones must be ignored, etc -- it depends a lot from the artist/dept/studio). - what you call tiny a file size may be not acceptable size for others. exporting a camera with a frame range spanning 1000 frames will result in 2mb on disk just to store a frame range which can be specify as simply as 1-1001x1. Time is also spent in reading those data. Size is just an argument as convenience could be, this makes the data dense and editing might be tedious as error-prone. We had a need once to export very dense geometries with their internal structure. Because our pipeline was not designed to handle so much density, the firt try has been a disaster resulting in easily an hour
Re: Open source json camera I/O platform
Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files.
Re: Open source json camera I/O platform
Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. --
Re: Open source json camera I/O platform
When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.comwrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are!
RE: Open source json camera I/O platform
I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. Matt From: softimage-boun...@listproc.autodesk.com [mailto:softimage-boun...@listproc.autodesk.com] On Behalf Of Tim Crowson Sent: Wednesday, July 10, 2013 5:34 PM To: softimage@listproc.autodesk.com Subject: Re: Open source json camera I/O platform This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.commailto:alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene. On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.commailto:raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale factors whether you like it or not, and the format will be rendered useless by not having control over those when it'll happen. There are things one omits or includes for the sake of forecfully encouraging good practices, those are practical-philosophical choices and are fine, and best made by a benevolent dictator for the project (any half decent programming language out there has craptons of those, and they are important to give languages and format identity and coherence). Scaling and units are not one of those, they are a fundamental requirement implicit to what you set off to describe with these files. -- -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are! --
Re: Open source json camera I/O platform
Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.com wrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data in the list but would simply read the values of all parameters at each frame and plot the same when I am reading the values. But what you are doing with keyframes value storage also works, in fact I think it reduces the file size considerably in case you do not have much keyframes in the source scene.** ** ** ** On Wed, Jul 10, 2013 at 7:41 PM, Raffaele Fragapane raffsxsil...@googlemail.com wrote: Units and scale, and how they correlate, are extremely important even in a light weight, portable format. Actually, probably more so in one than in a more complex scenario. You can't assume people will not care about those because their workflow will be independent of it like yours was for a few example productions, because very frequently they won't have the choice. As an interop format that deals with external contributions (rendering engines and so on being heavily dependent on it) you WILL bump into scale
Re: Open source json camera I/O platform
The unit needs only be an arbitrary value in the header. Autodesk can say whatever it wants, but the truth is that if you change maya from imperial to metric at the beginning of a project (and you might have that on client's side) there will be repercussions, and if your cameras were intended as 1cm but get imported as 1 inch things will be out of whack. Majorly. Several parameters, especially so if this will get a further level of abstraction later on, are actually world scale dependent. The flm back can change with a unit change (in some apps it does, in some it doesn't), several rendering and grooming parameters change and so on. As for scale, I've had plenty instances when the camera was scaled for various reasons, frequently enough to be relevant entire chunks of a pipe would rely on a stupid-renderman-trick style scaled camera. On Thu, Jul 11, 2013 at 2:10 PM, Gene Crucean emailgeneonthel...@gmail.comwrote: Thanks for the input guys! I'm ingesting all of it :) I'm quite against adding units into the main camera section of the file... but what about adding them to the metadata section? I really don't understand why anyone would want this in the file though. Units should only be conceptual imo. Autodesk says that 1 SI unit = 1 decimeter, but it has no concept of units... at all. Our current project is in meters, so conceptually we just know that 1 SI unit = 1 meter. Did we change anything? Nope. Same thing in Maya... 1 unit = 1 meter. Didn't change a thing on the Maya side either. I would love for someone to give me an example of why this should be different. Either way, I'll have an update tomorrow at some point, along with i/o for Houdini and updated Soft scripts. Maya is next and then hopefully I can talk one of our Nuke dev's into banging out an importer. Unless someone on here knows it's API and want's to donate some skills (once the 1.0 spec is finished). Same with any other apps :) Cheers On Wed, Jul 10, 2013 at 6:59 PM, Matt Lind ml...@carbinestudios.comwrote: I started a toolset a few years ago based on XML as well. It works and I can store robust data, but the downside is the file sizes are huge and slow to read/write. Memory becomes an issue at some point. If you only want to transfer cameras or simple stuff, it works fine, but large scenes with lots of animation data is not advised with XML as other formats may be better suited. ** ** Matt ** ** ** ** ** ** ** ** ** ** *From:* softimage-boun...@listproc.autodesk.com [mailto: softimage-boun...@listproc.autodesk.com] *On Behalf Of *Tim Crowson *Sent:* Wednesday, July 10, 2013 5:34 PM *To:* softimage@listproc.autodesk.com *Subject:* Re: Open source json camera I/O platform ** ** This is great to see! I have something similar in the wings that's only partially implemented, but in XML instead. It stores most of the stuff Jo was talking about. I wrote it as a way to export cameras and nulls to Nuke and Fusion. My goal is to have a series of tools for various apps all writing and reading a common XML file. Cameras and nulls can be exported, then updated if something changes. With custom UI for setting options. Anyway, I haven't finished writing all the different plugins, but I've got a couple of apps covered already. I debated between JSON and XML and finally just went with XML. Glad to see this in the works, Gene! Can't wait to see more! -Tim C On 7/10/2013 7:10 PM, Raffaele Fragapane wrote: When you'll have at most a few dozen curves, even on a thousand frame long sequence, I honestly don't think cheapening the data matters one iota. You can always introduce a zip compression stage to the I/O. Optimizing early and ending data poor is always a mistake. Purging is easy, both on I/O and in dev terms, adding data you don't have is usually betwene painful and downright impossible. If footprint was a concern here, sure, it'd make sense, on something that on a bad day will have a hundred parameters at the most (and for a mono cam I'd struggle to think of a hundred parameters I'd want animated) saving 16 floats per frame instead of 64 makes little difference in practical terms. ** ** On Thu, Jul 11, 2013 at 10:01 AM, Alok Gandhi alok.gandhi2...@gmail.com wrote: Hey Gene looking at your schema I do not see animated value for parameters like focal length, near and far planes. Though near and far are not usually keyed but you never know. I have worked on a stereoscopic project and we did need to plot the clipping planes. Anyways, focal length does fairly get animated at times. In the interest of generality and being generic I would make room for values for nearly all animatable parameters. For the case of optimization the writer plugin can use only one value in the list if the parameter is not animated else it will take all the key frame values. Also I would not care for the whole keyframe and tangent data
Open source json camera I/O platform
Hey folks, Who's in the mood for some open-source camera I/O code? I'm kind of getting bummed out on having to write the same camera tools at every studio just to get a simple, lightweight and most importantly reliable camera pushed around from app to app. FBX does *not* cut it, not to mention it's not available for all apps that could make use of it. So I thought I would whip up a spec based on json and offer it up open source so anyone willing to donate some time, could create some simple tools to import/export for their favorite app. The spec is VERY lightweight and doesn't include some things that I'm sure someone will want... but please let me know your thoughts. I already have a Softimage plugin working (consider it alpha). At this point it only has minor sanity checking and logic tests and I'm sure there are a zillion ways to improve it. But it's a start. The goal is to at least have plugins that all read and write the same spec from Houdini, Softimage, Maya, Max, Blender, Nuke... and more. I've built the Soft one using PyQt and it would be nice to maintain some consistency between apps, so I'm hopeful that the other versions could be based off of the same .ui file. What do you guys think? Any interest in this? I know it's a simple thing but I'm sure a lot of you also write these tools at studios quite a bit too and could possibly be into something like this. Check out the spec and source, and if you have time, play with the Soft plugin here: https://bitbucket.org/crewshin/json-cam If you have completely zero interest in this... no worries. Thanks for looking. -- -Gene www.genecrucean.com