Dear all,
While the dbt core integration as such is great, the lineage graph isn’t very useful. In a real world case, lineage graphs can become quite involved and the current graph layout algorithm is not able to cope with this. It would be great if there would be a more state of the art algorithm used like Sugiyama’s Algorithm. Moreover, more navigation options would be great.
Hi Glauber,
Thank you for your feedback on our lineage display!
Our team have already added your request to our backlog and will get back to you if there is any further update on it.
Best regards,
Hi Glauber,
For better understanding about your use case and what you are trying to achieve. Could you please elaborate more on how the current graph layout algorithm is not able to cope with the lineage graphs, as well as how you want your graph to be displayed.
These information will help us a lot in improving this.
Thank you.
Dear Duong Ly,
Thanks for your quick reply and adding this to your backlog. This is much appreciated!
In brevity, a more spacious layout with fewer edge crossings would be helpful. This would be great in combination with options to navigate over the canvas and zoom in and out.
The current graph layout algorithm seems to position nodes on a grid of equal distances. While this might look good for small lineage graphs with, say, 10 models, the graph becomes unreadable for more complex setups due to an overwhelming net of crossing edges.
There are graph layout algorithms that compute node positions that minimise the total number of edge crossings. A prominent example is Sugiyama’s Algorithm. In our experience this algorithm works quite well with data warehouse type of model dependencies used for BI.
